Aggeregate test summary files in CircleCI workflow runs #34989
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
.circleci/create_circleci_config.py
Outdated
| {"run": """while [[ $(curl --location --request GET "https://circleci.com/api/v2/workflow/$CIRCLE_WORKFLOW_ID/job" --header "Circle-Token: $CCI_TOKEN"| jq -r '.items[]|select(.name != "collection_job")|.status' | grep -c "running") -gt 0 ]]; do sleep 5; done"""}, | ||
| {"run": 'python utils/process_circleci_workflow_test_reports.py --workflow_id $CIRCLE_WORKFLOW_ID'}, |
There was a problem hiding this comment.
In circle-ci can't we just use something like with or require? It will automatically wait for the run_test to run, and then fetch results?
(It could even be a github action no?)
There was a problem hiding this comment.
Job 2 require Job 1: if job 1 failed, job 2 won't be triggered (sad 😢 ). when: always can't apply for jobs in a workflow (only steps in a job) for CircleCI.
That is why such wait is implemented (although I don't like it).
There was a problem hiding this comment.
OK then you can merge but I would like for this test to not block merge
There was a problem hiding this comment.
You mean if this new job fails, it shouldn't block the merge?
I will see if I can configure this 🤔
There was a problem hiding this comment.
|
|
||
| job_test_summaries = {} | ||
| for artifact in job_artifacts: | ||
| if artifact["path"].startswith("reports/") and artifact["path"].endswith("/summary_short.txt"): |
There was a problem hiding this comment.
I think using the junit.xlm will be more "foolproof" than regex parsing no?
There was a problem hiding this comment.
There is no regex in this script.
Regarding junit.xlm, yes probably more foolproof, but more post-process to be done I think:
xml's structure is more complex thensummary_short.txt.- it doesn't give something like
tests/generation/test_utils.py::GenerationIntegrationTests::test_generated_length_assisted_generationthat we could copy-paste and run directly.
summary_short.txt is working well in a very simple way. If we find xlm is necessary in the future, happy to make the change however.
|
Let's fix the test to merge! |
|
will merge on Monday instead :-) Thanks for approval |
What does this PR do?
See the new
collection_joboutputs in the artifacts tab.This could ease the searching if a test is run or not, run in which jobs, their status in each job etc.