Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RHOAIENG-17695: chore(ci): create a test for calling oc version in the test, which can be run with ci testing #829

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

RHOAIENG-17695: chore(ci): create a test for calling oc version in the test, which can be run with ci testing #829

wants to merge 1 commit into from

Conversation

jiridanek
Copy link
Member

https://issues.redhat.com/browse/RHOAIENG-17695

Description

Pytest! Yay!

How Has This Been Tested?

poetry run pytest tests/containers \
  --image ghcr.io/jiridanek/notebooks/workbench-images:runtime-minimal-ubi9-python-3.9-main_fe3641a01ac9e1e8f694f1a8bce3b34cacf9ae57 \
  --image quay.io/modh/odh-minimal-notebook-container@sha256:eec50e5518176d5a31da739596a7ddae032d73851f9107846a587442ebd10a82 \
  --image=ghcr.io/jiridanek/notebooks/workbench-images:base-ubi9-python-3.11-jd_try_oc_command_cleanup_b1b8d1f10a329317d85f0ea54580e170b002cae8

This should fail one test, because one of the images is the one that was seen failing on FIPS systems.

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

@jiridanek
RHOAIENG-17695: chore(ci): create a test for calling oc version in …
b7c937e 
…the test, which can be run with ci testing
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 8, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jiridanek. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jiridanek
jiridanek commented Jan 8, 2025
View reviewed changes
Copy link
Member Author

@jiridanek jiridanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design issues

  • suite should support local run
  • github run
  • and on openshift run

Suite should handle reporting of failing tests

  • junit, or markdown table, something that's clear at a glanc

Makefile tests should be rewritten into this

  • the source of truth should be the manifest files
  • need to figure out which manifest applies for which image

.github/workflows/build-notebooks-TEMPLATE.yaml
env:
DOCKER_HOST: "unix:///var/run/podman/podman.sock"
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE: "/var/run/podman/podman.sock"
RYUK_CONTAINER_IMAGE: "testcontainers/ryuk:0.8.1"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jstourac suggested disabling ryuk for gha runs

tests/containers/base_image_test.py
"""Tests that are applicable for all images we have in this repository."""

def test_oc_command_runs(self, image: str):
container = testcontainers.core.container.DockerContainer(image=image, user=123456, group_add=[0])
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jstourac suggested abstracting this object away, so the same test can run against local docker as well as remote kubernetes, depending on some cli switch or some other configuration mechanism

tests/containers/base_image_test.py
logging.debug(output.decode())
assert ecode == 0

def test_oc_command_runs_fake_fips(self, image: str, subtests: pytest_subtests.SubTests):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jstourac suggested skipping this test when running suite on openshift, openshift will test real configurations, not these fake ones

tests/containers/conftest.py
metafunc.parametrize(image.__name__, metafunc.config.getoption("--image"))


# https://docs.pytest.org/en/stable/how-to/fixtures.html#parametrizing-fixtures
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jstourac was discouraged by this long doc page and left the office for today shortly after opening it

tests/containers/base_image_test.py
import pytest_subtests


class TestBaseImage:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jstourac wondered how to structure the tests when more are to be added, like when the version tests are rewritten from makefile to here

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 8, 2025

@jiridanek: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/images b7c937e link true /test images

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

This was referenced Jan 9, 2025
Open
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andyatmiami andyatmiami Awaiting requested review from andyatmiami

@caponetto caponetto Awaiting requested review from caponetto

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jiridanek