diff --git a/docs/BUG_REPORT.md b/docs/BUG_REPORT.md new file mode 100644 index 00000000000000..1620aa154f9d79 --- /dev/null +++ b/docs/BUG_REPORT.md @@ -0,0 +1,109 @@ +# Reporting bugs + +## Writing an effective bug report + +When reporting a bug, start with the question +`What does a bug report need to tell the developer`. + +Generally you want the following parts covered: + +- What is the problem + +- How can the developer reproduce the problem (to see it for themselves), to + bisect when it was introduced or to find if it got fixed already. + +- At what point does the problem occur + +- What environment did this occur in + +Make sure the above items are covered and the bug is easy to review and parse: + +- **Title** should clearly describe the problem. Bugs are often sorted from + the issue list which only contains the title + +- **Logs** should generally be attachments (drag & drop or click on bottom bar + when entering issue text) and not inline with the issue. + +- **Reproduction steps** and **environment** should be clearly highlighted. If + running commands reproduce the issue (very common), the commands should be + in a code block/script format. + +### Describing the problem + +Make sure the issue is obvious or provide a link explaining why the expected +result is not met. + +Examples: + +- `(Core dump) seen` is obvious since there should be no core dumps + +- `Failure trying to read attribute X in cluster Y which is marked MANDATORY in the spec` + references the spec and describes why attribute read should succeed. + +- `Failure trying to write attribute X in cluster Y, which is enabled since cluster FeatureMap enabled X and spec describes as writable.` + references the spec and explicitly states that an optional attribute is + enabled based on device status + +- `Running certification test TC-A-B-C (link included) fails at step 3: test case asks for command to succeed, I get ACCESS_DENIED instead` + describes a pre-defined test case that is expected to pass but fails. Note + that full link to the test description is needed (and should be covered by + 'how to reproduce' part) + +Unless manually curated (e.g. few lines showing the problem), logs should be +always attachments and not inlined in the bug as the make the bug report too +long. + +### Reproduction steps and when does the issue occur + +Include all steps needed to reproduce the problem. Link any supporting +documentation. + +If stating something of the form `TC-A-B-C step 4 fails` then there should be a +link to TC-A-B-C and ideally a list of the commands of each step since test +cases may change over time. + +The bug report should contain all the information for a developer to reproduce +the issue without needing to have access to CHIP Test case repository (not +everyone does) + +### Environment for reproduction + +Assume that the developer will want to reproduce the issue and will try to +mirror your environment to a reasonable degree. For this, at a minimum the +platforms on which everything is running is needed. + +Try to provide as much information as seems relevant. At a minimum this could +look like +`Failed to commission nrf board using chip-tool running on linux. Used build on SHA abcde...`. +This provides basic information (use nrf board, use chip-tool on linux, default +build) to get started. Beyond that, you can refine if more items seem relevant: + +- `Tested on TE9` or `Tested on interop branch xyz` gives a build reference + point. Useful when branches/tags are used instead of master branch as + development happens on master branch. + +- `Thread devices fail, tested with qpg and efr32` shows that this seems to be + a general thread issue and developer can investigate on multiple of them + +* `Tested with avahi-build and it passes/fails` helps the developer with + information of non-default builds that pass/fail to narrow down the problem + +* `Passes with chip-tool-darwin and repl but fails with chip-tool` helps the + developer in narrowing down the problem + +### Additional information + +Providing additional information that can be helpful is encouraged. Each bug +report is different here. Some examples: + +- `This worked last week (around Jan 5) but started failing in recent master builds` + +- `Specification changed this attribute from optional to mandatory so this may be the cause of the issue` + +- `This issue may be related to #1234 as the same error is seen, however the reproduction steps seemed distinct enough that I openend a new issue` + +- `While running this, I observed 100% CPU before the operation finally timed out` + +- `While running this test, I observed device under test rebooting, logs attached.` + +- `This only happens intermitently - I see it about 30% of the time`