Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Run all sub-project tests in test_pack_doc.yml workflow #1382

Merged
merged 10 commits into from
Apr 11, 2023
Merged

fix: Run all sub-project tests in test_pack_doc.yml workflow #1382

merged 10 commits into from
Apr 11, 2023

Conversation

bdferris-v2
Copy link
Collaborator

@bdferris-v2 bdferris-v2 commented Apr 7, 2023 •
edited
Loading

As noted in #1381, the test_pack_doc.yml is not currently running all of the project's unit-tests, which is problematic if we want to be sure new PRs aren't introducing test breakages.

To address the issue, this PR does the following:

  • Upgrade to Gradle 7.4 (currently at 7.2), which introduces a new Test Report Aggregation plugin. This plugin offers better support for test aggregation across sub-projects.
  • Refactor the project to use the testAggregateTestReport support. We do this in a way where all sub-projects are automatically added for report aggregation. Previously, each project had to be hand-configured, which caused us to miss a number of new projects as they were added (including output-comparator, per the bug).
  • Fix all the tests that were actually failing and we didn't know about it (mostly on Windows).
  • I also took this opportunity to move a bunch of repeated unit-test configuration from individual build.gradle files and specify it once in the root project.
  • Run the unit tests with gradle test to make sure you didn't break anything
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@bdferris-v2
Upgrade to Gradle 7.4 so that we can use the new `testReportAggregati…
e53bef7 
…on`, which is a more standard way of aggregating test reports across all sub-projects. Use this opportunity to consolidate a bunch of test options into the root project for consistent configuration.
@bdferris-v2
Comment tweak.
139d140
@bdferris-v2
Restore code to make sure copyRulesMarkdown runs before test execut…
0b5b8f3 
…ion.
@bdferris-v2
Make sure affected sources are in consistent sorted order for repeata…
a8952b6 
…ble test execution.
@bdferris-v2
Disable UI tests in headless environments.
807d8a8
@github-actions
Copy link
Contributor

github-actions bot commented Apr 7, 2023

✅ Rule acceptance tests passed.
New Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
New Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
0 out of 1425 sources (~0 %) are corrupted.
Commit: 989023d
Download the full acceptance test report here (report will disappear after 90 days).
✅ Rule acceptance tests passed.

@bdferris-v2 bdferris-v2 requested a review from davidgamez April 7, 2023 06:28
@bdferris-v2 bdferris-v2 linked an issue Apr 7, 2023 that may be closed by this pull request
output-comparator sub-module is not being included in test_pack_doc.yml workflows #1381
Closed
@github-actions
Copy link
Contributor

github-actions bot commented Apr 7, 2023

✅ Rule acceptance tests passed.
New Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
New Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
0 out of 1425 sources (~0 %) are corrupted.
Commit: dd3b81e
Download the full acceptance test report here (report will disappear after 90 days).
✅ Rule acceptance tests passed.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 7, 2023

✅ Rule acceptance tests passed.
New Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
New Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
0 out of 1425 sources (~0 %) are corrupted.
Commit: 433756e
Download the full acceptance test report here (report will disappear after 90 days).
✅ Rule acceptance tests passed.

@github-actions
Copy link
Contributor

✅ Rule acceptance tests passed.
New Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Errors: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
New Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Warnings: 0 out of 1425 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
0 out of 1425 sources (~0 %) are corrupted.
Commit: 14a2ba0
Download the full acceptance test report here (report will disappear after 90 days).
✅ Rule acceptance tests passed.

@bdferris-v2 bdferris-v2 merged commit f2f43e3 into master Apr 11, 2023
@bdferris-v2 bdferris-v2 deleted the issue/1381/all_tests branch April 11, 2023 16:36
@davidgamez davidgamez mentioned this pull request Apr 13, 2023
Merged
4 tasks
bradyhunsaker pushed a commit to bradyhunsaker/gtfs-validator that referenced this pull request Apr 25, 2023
@bdferris-v2 @bradyhunsaker
fix: Run all sub-project tests in test_pack_doc.yml workflow (Mobil…
95f2f58 
…ityData#1382)

* Upgrade to Gradle 7.4 so that we can use the new `testReportAggregation`, which is a more standard way of aggregating test reports across all sub-projects.  Use this opportunity to consolidate a bunch of test options into the root project for consistent configuration.

* Comment tweak.

* Restore code to make sure `copyRulesMarkdown` runs before test execution.

* Make sure affected sources are in consistent sorted order for repeatable test execution.

* Disable UI tests in headless environments.

* Fix failing test on Windows due to line separators.

* Attempt MobilityData#2 to test correctly under Windows.

* Consolidate Java compile options for consistency across projects.  This will also help some tests pass on Windows environments.

* Get another test passing on Windows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidgamez davidgamez davidgamez approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

output-comparator sub-module is not being included in test_pack_doc.yml workflows
2 participants
@bdferris-v2 @davidgamez