Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Add failing/flaking test issue templates to the repository #7778

Conversation

furkatgofurov7
Copy link
Member

@furkatgofurov7 furkatgofurov7 commented Dec 19, 2022

What this PR does / why we need it:
While we can still modify existing issue templates when reporting CI related issues, it would be very helpful if we could have issue templates ready while opening failing/flaking test issues in the repository (especially handy for relase team CI signal members) similar to the one used in the k8s repo, as IMO it helps to:

  • gather all the required and additional information needed to tackle the issue regardless of whoever opens it
  • standardize the issues of the same kind in the future

Note: I am still using the markdown formatted issue templates for new failing/flaking-test templates in this PR but we have a possibility to move all existing templates from markdown format to GitHub forms for issue templates which basically YAML representation of the template.
Example issue template in kubernetes repo to get the idea how it looks like: https://github.com/kubernetes/kubernetes/issues/new?assignees=&labels=kind%2Ffailing-test&template=failing-test.yaml
k/k repo has already switched to the GitHub forms in kubernetes/kubernetes#104468. We can replace all templates with the GitHub forms if agreed and if not we could still stick to the markdown format as well (since the feature itself seems still in beta).

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 19, 2022
Copy link
Contributor

@killianmuldoon killianmuldoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good idea and something successive release teams can refine over time. Can we also add a task for the release team (maybe as the responsibility of the CI lead but the team can decide that) to maintain and review these templates?

I'm happy with adding them, but it's a good idea to have a defined role for maintaining and keeping these up to date as needed.

@furkatgofurov7 furkatgofurov7 force-pushed the add-flaking-failing-test-templates/furkat branch from 2136a90 to c4d70ab Compare December 20, 2022 13:02
@furkatgofurov7
Copy link
Member Author

furkatgofurov7 commented Dec 20, 2022

I think this is a good idea and something successive release teams can refine over time. Can we also add a task for the release team (maybe as the responsibility of the CI lead but the team can decide that) to maintain and review these templates?

I'm happy with adding them, but it's a good idea to have a defined role for maintaining and keeping these up to date as needed

@killianmuldoon thanks for the review, I totally agree with the point. I have added it as part of the CI lead responsibilities which makes more sense IMO, PTAL, but I will wait for others' opinions as well.

/cc @fabriziopandini
@kubernetes-sigs/cluster-api-release-team

@furkatgofurov7 furkatgofurov7 force-pushed the add-flaking-failing-test-templates/furkat branch 2 times, most recently from 337d574 to e9592f4 Compare December 20, 2022 18:48
Copy link
Contributor

@ykakarap ykakarap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea. 👍🏼
We can refine the template over time as we start using them but this is a great start.

.github/ISSUE_TEMPLATE/failing-test.md Outdated Show resolved Hide resolved
.github/ISSUE_TEMPLATE/flaking-test.md Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added kind/flake Categorizes issue or PR as related to a flaky test. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. labels Dec 22, 2022
@furkatgofurov7 furkatgofurov7 force-pushed the add-flaking-failing-test-templates/furkat branch from e9592f4 to 989c7b5 Compare December 22, 2022 09:49
@furkatgofurov7
Copy link
Member Author

furkatgofurov7 commented Dec 22, 2022

Labels were added automatically after review comment, removing them.
/remove-kind flake

@k8s-ci-robot k8s-ci-robot removed the kind/flake Categorizes issue or PR as related to a flaky test. label Dec 22, 2022
@furkatgofurov7
Copy link
Member Author

/remove-kind failing-test

@k8s-ci-robot k8s-ci-robot removed the kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. label Dec 22, 2022
@furkatgofurov7 furkatgofurov7 changed the title 🌱 Add failing-/flaking- test templates to the repository 🌱 Add failing/flaking test issue templates to the repository Dec 22, 2022
@fabriziopandini
Copy link
Member

Let's give it a try!
/lgtm
/approve

Also +1 from my side to switch to GitHub forms in a follow-up PR

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 29, 2022
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: bbcee8bea0c0df9b44dfc9eb52b810c09ad5a0df

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 29, 2022
@k8s-ci-robot k8s-ci-robot merged commit 9b70157 into kubernetes-sigs:main Dec 29, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.4 milestone Dec 29, 2022
@furkatgofurov7
Copy link
Member Author

Also +1 from my side to switch to GitHub forms in a follow-up PR

@fabriziopandini great to hear that, I created a separate issue #7815 to track it.

@CecileRobertMichon
Copy link
Contributor

I'm late to the party but I love this, thanks for doing this @furkatgofurov7 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants