Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ReporterTests: Create / use a minimal test asset covering all important cases #2287

Closed
fviernau opened this issue Feb 28, 2020 · 7 comments
Closed
Labels
enhancement Issues that are considered to be enhancements reporter About the reporter tool

Comments

@fviernau
Copy link
Member

fviernau commented Feb 28, 2020

The rather higher level tests for the reporters lack coverage for actually quite important cases. E.g. the NoticeSummaryReporter does not cover path excludes. Further the tests use test assets shared with other test so that they cannot be easily adjusted.

Let's create on test asset which covers all cases relevant test cases for reporters and use for all the reporter tests to get more meaningful test and a better use case coverage.

@sschuberth
Copy link
Member

E.g. the notice reporter does not cover path excludes.

Is this still true now that I see that #2286 did need to also update expected test results for the NOTICE-BY-PACKAGE reporter?

@fviernau
Copy link
Member Author

Is this still true now that I see that #2286 did need to also update expected test results for the NOTICE-BY-PACKAGE reporter?

Thanks, I've updated the description accordingly.

@sschuberth
Copy link
Member

@sschuberth
Copy link
Member

Reopening because @mnonnenmacher says:

Not all reporter tests use the TestData yet, and some features like scope excludes or license configurations are not tested with the reporters.

@sschuberth sschuberth reopened this Oct 5, 2020
@sschuberth
Copy link
Member

This is somewhat related to #2842.

@sschuberth sschuberth added enhancement Issues that are considered to be enhancements reporter About the reporter tool labels Oct 24, 2022
@mnonnenmacher
Copy link
Member

This is also related to #5991, as an abstraction for the OrtResult could make it easier to use mocks in some tests without having to create a complete OrtResult.

@sschuberth
Copy link
Member

Closed as part of backlog grooming. Feel free to comment if you would like to contribute to this.

@sschuberth sschuberth closed this as not planned Won't fix, can't repro, duplicate, stale Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Issues that are considered to be enhancements reporter About the reporter tool
Projects
None yet
Development

No branches or pull requests

3 participants