Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TEP-0079] Catalog Automated Testing and Tekton Dogfooding #914

Merged
merged 1 commit into from
Jan 23, 2023
Merged

[TEP-0079] Catalog Automated Testing and Tekton Dogfooding #914

merged 1 commit into from
Jan 23, 2023

Conversation

QuanZhang-William
Copy link
Member

@QuanZhang-William QuanZhang-William commented Jan 4, 2023 •
edited
Loading

This commit updates the "Automated Testing and Dogfooding" section of TEP-0079, which adds the testing strategy proposal for the Verified Catalogs and a testing infra template for Community Catalogs.

This commit also update the tep from proposed to implementable 🥂

@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 4, 2023
@QuanZhang-William
Copy link
Member Author

QuanZhang-William commented Jan 4, 2023 •
edited
Loading

/kind tep

1 similar comment
@QuanZhang-William
Copy link
Member Author

/kind tep

@tekton-robot tekton-robot added the kind/tep Categorizes issue or PR as related to a TEP (or needs a TEP). label Jan 4, 2023
@QuanZhang-William
Copy link
Member Author

/cc @jerop @dibyom @lbernick @vdemeester @vinamra28

vinamra28
vinamra28 reviewed Jan 9, 2023
View reviewed changes
Copy link
Member

@vinamra28 vinamra28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 9, 2023
@pritidesai
Copy link
Member

/assign @dibyom

@pritidesai
Copy link
Member

@vinamra28 please approve the proposal instead of lgtm based on our guidelines.

dibyom
dibyom reviewed Jan 11, 2023
View reviewed changes
teps/0079-tekton-catalog-support-tiers.md Outdated Show resolved Hide resolved
teps/0079-tekton-catalog-support-tiers.md Outdated

#### Community Catalog Template Testing Infrastructure

The test infrastructure in the plumbing repo is designed for all `tektoncd` projects, which may not fit a Community Catalog repo's needs and may be too complicated for the authors to start with. To encourage Community Catalog authors to set up automated tests for their catalogs (using Tekton!), we propose to provide a template repo in the `tektoncd-catalog` GitHub org for Community Catalog authors to follow. This tutorial-like template should contain the **simple and minimum** required setup for the automated build tests and integration tests, guiding Community Catalog authors to achieve the same testing functionality as the Verified Catalogs using vanilla Tekton primitives.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we suggesting each project have their own eventlistener?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the choice is on the community catalog authors? We just provide an example to them to start with. But I would personally use the same el for multiple projects.

jerop
jerop reviewed Jan 12, 2023
View reviewed changes
Copy link
Member

@jerop jerop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @QuanZhang-William!

There are some open questions around the testing strategy:

  • how often would we run the integration tests? nightly? on PRs only?
  • which Tekton versions would we test these resources against? nightly release? latest release? minimum compatible version?

@bobcatfish discussed some ideas here: #170

/assign

teps/0079-tekton-catalog-support-tiers.md Outdated Show resolved Hide resolved
teps/0079-tekton-catalog-support-tiers.md Outdated Show resolved Hide resolved
@jerop
Copy link
Member

jerop commented Jan 12, 2023

putting a hold so that it's not accidentally merged on the first approval because it already has an lgtm

/hold

@tekton-robot tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 12, 2023
@QuanZhang-William QuanZhang-William force-pushed the tep-0079-automated-testing-and-dogfooding branch from cfff349 to 60d7621 Compare January 12, 2023 21:41
@tekton-robot tekton-robot removed the lgtm Indicates that a PR is ready to be merged. label Jan 12, 2023
@QuanZhang-William
Copy link
Member Author

Thank you @QuanZhang-William!

There are some open questions around the testing strategy:

  • how often would we run the integration tests? nightly? on PRs only?
  • which Tekton versions would we test these resources against? nightly release? latest release? minimum compatible version?

@bobcatfish discussed some ideas here: #170

/assign

Thanks @jerop for the link! I have added 2 sections under the Integration Test section, talking about the nightly integration test and PR merge integration test respectively, including the Tekton Pipelines release that will be used. PTAL 😁

@QuanZhang-William QuanZhang-William force-pushed the tep-0079-automated-testing-and-dogfooding branch from 60d7621 to 557fb7c Compare January 13, 2023 19:48
@QuanZhang-William QuanZhang-William mentioned this pull request Jan 13, 2023
Merged
jerop
jerop approved these changes Jan 17, 2023
View reviewed changes
Copy link
Member

@jerop jerop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @QuanZhang-William!

teps/0079-tekton-catalog-support-tiers.md Show resolved Hide resolved
@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 17, 2023
@jerop
Copy link
Member

jerop commented Jan 17, 2023

/hold cancel

@dibyom @vinamra28 please take a look

@tekton-robot tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 17, 2023
@QuanZhang-William
[TEP-0079] Catalog Automated Testing and Dogfooding
028a235 
This commit updates the "Automated Testing and Dogfooding" section of TEP-0079, which adds the testing strategy proposal for the Verified Catalogs and a testing infra template for Community Catalogs.
@QuanZhang-William QuanZhang-William force-pushed the tep-0079-automated-testing-and-dogfooding branch from 557fb7c to 028a235 Compare January 17, 2023 16:06
@tekton-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dibyom, jerop, vinamra28

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@afrittoli
Copy link
Member

Approved during API WG
/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 23, 2023
@tekton-robot tekton-robot merged commit cc963dc into tektoncd:main Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vinamra28 vinamra28 vinamra28 approved these changes

@jerop jerop jerop approved these changes

@dibyom dibyom dibyom approved these changes

@mnuttall mnuttall Awaiting requested review from mnuttall

@pratap0007 pratap0007 Awaiting requested review from pratap0007

@lbernick lbernick Awaiting requested review from lbernick

@vdemeester vdemeester Awaiting requested review from vdemeester

Assignees

@afrittoli afrittoli

@dibyom dibyom

@jerop jerop

@vinamra28 vinamra28

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/tep Categorizes issue or PR as related to a TEP (or needs a TEP). lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
TEP Steward Assignments
Status: UnAssigned
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@QuanZhang-William @pritidesai @jerop @tekton-robot @afrittoli @dibyom @vinamra28