TELCOV10N-452 IMPL junit2obj filter plugin #510

mvk · 2024-12-18T19:52:29Z

make sure all the relevant TestSuite & TestCase attributes are properly passed into the json object
TODO: migrate to it in other contexts

SUMMARY

There is currently a similar filter named junit2dict
It does not take all the attributes from TestCase and TestSuite.
This filter converts JUnit XML reports to JSON format without losing attributes.
Hopefully it satisfies current junit2dict users so they can start using this one.

The safe migration process is proposed:

use this PR in current reporting context, safely not breaking any code still using junit2dict
stabilize json2obj, add tests and evaluate whether replacing junit2dict can be done
once ready - move json2obj to json2dict

ISSUE TYPE

Enhanced Feature

Tests

Need help writing tests for a filter plugin.
I only verified the code is doing what it is supposed to, but did write any unit tests.
The code using this filter is not deployed yet, but it requires the filter.

TestDallasWorkload: preflight-green

dcibot · 2024-12-18T19:52:34Z

from change #510:

no check (not a code change)

dcibot · 2024-12-18T19:52:35Z

from change #510:

no check (not a code change)

softwarefactory-project-zuul · 2024-12-18T19:56:06Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/1054a5f518b749c49d72281876900863

✔️ dci-rpm-build-el8 SUCCESS in 2m 48s
✔️ dci-rpm-build-el9 SUCCESS in 2m 51s

ramperher · 2024-12-19T11:08:24Z

Hi @mvk , thanks for your contribution! I wanted to ask you, because in the PR, you are mentioning json2obj and json2dic, are they new filters to be included? Or are you referring to junit2obj and junit2dict?

Also, do you have any output of local tests where you tested this new filter?

Adding @tkrishtop in the look as she developed the junit2dict filter,.

tkrishtop · 2024-12-19T11:17:27Z

Hello guys, json2dict filter is to be deprecated, we plan to move all Junit parsing into dci control server. cc: @fredericlepied @ylamgarchal

dcibot · 2025-01-12T08:20:59Z

from change #510:

no check (not a code change)

dcibot · 2025-01-12T08:20:59Z

from change #510:

no check (not a code change)

softwarefactory-project-zuul · 2025-01-12T08:24:03Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/37ef8edf126547c090989f7c32763056

✔️ dci-rpm-build-el8 SUCCESS in 2m 52s
✔️ dci-rpm-build-el9 SUCCESS in 2m 29s

dcibot · 2025-01-12T11:33:02Z

from change #510:

no check (not a code change)

dcibot · 2025-01-12T11:33:03Z

from change #510:

no check (not a code change)

softwarefactory-project-zuul · 2025-01-12T11:36:19Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/129c7a2737a34b0bab7774f3170e3e66

✔️ dci-rpm-build-el8 SUCCESS in 2m 45s
✔️ dci-rpm-build-el9 SUCCESS in 2m 42s

mvk · 2025-01-12T11:36:34Z

@ramperher hi.
Just wanted to safely add converting of junit xml files to a json string, the older filter (junit2dict) was missiung some attributes, and I have added some tests.

dcibot · 2025-01-12T11:57:01Z

from change #510:

no check (not a code change)

dcibot · 2025-01-12T11:57:02Z

from change #510:

no check (not a code change)

softwarefactory-project-zuul · 2025-01-12T12:00:07Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/8888949788484cd7a6958ed007b67720

✔️ dci-rpm-build-el8 SUCCESS in 2m 43s
✔️ dci-rpm-build-el9 SUCCESS in 2m 26s

dcibot · 2025-01-12T14:54:47Z

from change #510:

no check (not a code change)

dcibot · 2025-01-12T14:54:47Z

from change #510:

no check (not a code change)

softwarefactory-project-zuul · 2025-01-12T14:58:01Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/7878f2fec5bd47dfb43a23bc6575d01f

✔️ dci-rpm-build-el8 SUCCESS in 2m 55s
✔️ dci-rpm-build-el9 SUCCESS in 2m 46s

dcibot · 2025-01-12T21:32:48Z

from change #510:

no check (not a code change)

dcibot · 2025-01-12T21:32:48Z

from change #510:

no check (not a code change)

softwarefactory-project-zuul · 2025-01-12T21:35:59Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/eecd037ea1664499b7132067dd2419c2

✔️ dci-rpm-build-el8 SUCCESS in 2m 50s
✔️ dci-rpm-build-el9 SUCCESS in 2m 38s

dcibot · 2025-01-12T21:47:59Z

from change #510:

no check (not a code change)

dcibot · 2025-01-12T21:48:00Z

from change #510:

no check (not a code change)

softwarefactory-project-zuul · 2025-01-12T21:51:07Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/9cbff5fceb564bde98059be0e445a523

✔️ dci-rpm-build-el8 SUCCESS in 2m 50s
✔️ dci-rpm-build-el9 SUCCESS in 2m 35s

softwarefactory-project-zuul · 2025-02-25T11:03:49Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/305f9174d4084d77bf7c7dfd33304645

✔️ dci-rpm-build-el8 SUCCESS in 2m 49s
✔️ dci-rpm-build-el9 SUCCESS in 2m 26s

softwarefactory-project-zuul · 2025-02-25T11:13:09Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/0d9cd238225341d3ab6d76a634c57358

✔️ dci-rpm-build-el8 SUCCESS in 2m 44s
✔️ dci-rpm-build-el9 SUCCESS in 2m 37s

softwarefactory-project-zuul · 2025-02-25T11:19:58Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/1b7702745f3a4ab5a09b29e074c1872e

✔️ dci-rpm-build-el8 SUCCESS in 2m 53s
✔️ dci-rpm-build-el9 SUCCESS in 2m 39s

dcibot · 2025-02-25T11:51:45Z

from change #510:

SUCCESS https://www.distributed-ci.io/jobs/f7c6a43d-845f-4265-a4d8-b8460bdd4863/jobStates

tkrishtop · 2025-02-25T12:06:49Z

retest

plugins/filter/junit2obj.py

dcibot · 2025-02-25T12:44:05Z

from change #510:

SUCCESS https://www.distributed-ci.io/jobs/9af9bf80-2544-4d38-a93b-78b5aba83e06/jobStates

softwarefactory-project-zuul · 2025-02-25T14:13:17Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/a67f0954b9d64f3792ac8aa0e5049061

✔️ dci-rpm-build-el8 SUCCESS in 2m 45s
✔️ dci-rpm-build-el9 SUCCESS in 2m 20s

tkrishtop · 2025-02-25T14:42:30Z

Hi @mvk sorry, I see that Python2 compatibility is required by our CI, let's bring those back:

## Regressions are listed below
-ERROR: plugins/filter/junit2obj.py: missing: from __future__ import (absolute_import, division, print_function)
-ERROR: plugins/filter/junit2obj.py: missing: __metaclass__ = type

Also please remove crucible team from the reviewers, add verified signature to the commit, and I believe we can merge.

dcibot · 2025-02-25T14:47:33Z

from change #510:

SUCCESS https://www.distributed-ci.io/jobs/7c02a148-ec26-43fd-839b-5305063aed76/jobStates

softwarefactory-project-zuul · 2025-02-25T15:16:19Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/264f46050e334e7f9148faf527064cd8

✔️ dci-rpm-build-el8 SUCCESS in 2m 50s
✔️ dci-rpm-build-el9 SUCCESS in 2m 32s

mvk · 2025-02-25T15:18:11Z

Hi @mvk sorry, I see that Python2 compatibility is required by our CI, let's bring those back:
## Regressions are listed below
-ERROR: plugins/filter/junit2obj.py: missing: from __future__ import (absolute_import, division, print_function)
-ERROR: plugins/filter/junit2obj.py: missing: __metaclass__ = type
Also please remove crucible team from the reviewers, add verified signature to the commit, and I believe we can merge.

done.

tkrishtop

LGTM, please sign the commit and let's merge

Motivation: =========== It is more convenient to navigate data in splunk via json. The existing filter `redhatci.ocp.json2dict` misses multiple ibutes, has no tests, so this one collects more of them to properly into the resulting JSON object + is tested TODO: migrate to it in other contexts Details: -------- 1. Implement the filter `redhatci.ocp.junit2obj` 1. Add tools configuration for `pylint`, `pycodestyle`, et al. under `pyproject.toml` 1. Add `.gitignore` with `.docsible` 1. Implement unit test for the filter * fixtures under `tests/unit/data` * Add example data + output to the filter module docstring * test requirements under `tests/unit/requirements.txt` Signed-off-by: Maxim Kovgan <makovgan@redhat.com>

Signed-off-by: Maxim Kovgan <makovgan@redhat.com>

softwarefactory-project-zuul · 2025-02-25T15:39:12Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/78b211316c134751864193655041a930

✔️ dci-rpm-build-el8 SUCCESS in 2m 50s
✔️ dci-rpm-build-el9 SUCCESS in 2m 10s

tonyskapunk · 2025-02-25T16:29:43Z

retest

tonyskapunk · 2025-02-25T16:31:02Z

retest

dcibot · 2025-02-25T17:09:02Z

from change #510:

SUCCESS https://www.distributed-ci.io/jobs/c26c8ea3-cbd0-4fb6-b929-5bef40ea0d95/jobStates

mvk requested a review from a team as a code owner December 18, 2024 19:52

mvk force-pushed the mvk/filter_junit2obj branch from 95b6e42 to 5d69be5 Compare January 12, 2025 08:20

mvk force-pushed the mvk/filter_junit2obj branch from 7254edf to 46462b6 Compare January 12, 2025 21:32

mvk requested review from a team as code owners January 12, 2025 21:32

mvk force-pushed the mvk/filter_junit2obj branch from 46462b6 to 9090e32 Compare January 12, 2025 21:47

mvk force-pushed the mvk/filter_junit2obj branch from 1677054 to a9c9877 Compare February 25, 2025 11:09

mvk force-pushed the mvk/filter_junit2obj branch from a9c9877 to e01ee39 Compare February 25, 2025 11:16

tkrishtop requested changes Feb 25, 2025

View reviewed changes

plugins/filter/junit2obj.py Show resolved Hide resolved

mvk force-pushed the mvk/filter_junit2obj branch from ce0f7ef to ff5e03f Compare February 25, 2025 15:11

tkrishtop removed the request for review from a team February 25, 2025 15:31

tkrishtop approved these changes Feb 25, 2025

View reviewed changes

mvk added 2 commits February 25, 2025 17:35

upd after cr from @tkrishto

61438c5

Signed-off-by: Maxim Kovgan <makovgan@redhat.com>

mvk force-pushed the mvk/filter_junit2obj branch from ff5e03f to 61438c5 Compare February 25, 2025 15:36

tonyskapunk approved these changes Feb 25, 2025

View reviewed changes

mvk added this pull request to the merge queue Feb 25, 2025

Merged via the queue into redhatci:main with commit 5f595f3 Feb 25, 2025
7 checks passed

mvk deleted the mvk/filter_junit2obj branch February 25, 2025 18:20

mvk mentioned this pull request Mar 5, 2025

the role junit2json #591

Open

TELCOV10N-452 IMPL junit2obj filter plugin #510

TELCOV10N-452 IMPL junit2obj filter plugin #510

Conversation

mvk commented Dec 18, 2024 • edited by tkrishtop Loading

SUMMARY

ISSUE TYPE

Tests

dcibot commented Dec 18, 2024

dcibot commented Dec 18, 2024

softwarefactory-project-zuul bot commented Dec 18, 2024

ramperher commented Dec 19, 2024

tkrishtop commented Dec 19, 2024

dcibot commented Jan 12, 2025

dcibot commented Jan 12, 2025

softwarefactory-project-zuul bot commented Jan 12, 2025

dcibot commented Jan 12, 2025

dcibot commented Jan 12, 2025

softwarefactory-project-zuul bot commented Jan 12, 2025

mvk commented Jan 12, 2025 • edited Loading

dcibot commented Jan 12, 2025

dcibot commented Jan 12, 2025

softwarefactory-project-zuul bot commented Jan 12, 2025

dcibot commented Jan 12, 2025

dcibot commented Jan 12, 2025

softwarefactory-project-zuul bot commented Jan 12, 2025

dcibot commented Jan 12, 2025

dcibot commented Jan 12, 2025

softwarefactory-project-zuul bot commented Jan 12, 2025

dcibot commented Jan 12, 2025

dcibot commented Jan 12, 2025

softwarefactory-project-zuul bot commented Jan 12, 2025

softwarefactory-project-zuul bot commented Feb 25, 2025

softwarefactory-project-zuul bot commented Feb 25, 2025

softwarefactory-project-zuul bot commented Feb 25, 2025

dcibot commented Feb 25, 2025

tkrishtop commented Feb 25, 2025

dcibot commented Feb 25, 2025

softwarefactory-project-zuul bot commented Feb 25, 2025

tkrishtop commented Feb 25, 2025 • edited Loading

dcibot commented Feb 25, 2025

softwarefactory-project-zuul bot commented Feb 25, 2025

mvk commented Feb 25, 2025

tkrishtop left a comment

Choose a reason for hiding this comment

softwarefactory-project-zuul bot commented Feb 25, 2025

tonyskapunk commented Feb 25, 2025

tonyskapunk commented Feb 25, 2025

dcibot commented Feb 25, 2025

mvk commented Dec 18, 2024 •

edited by tkrishtop

Loading

mvk commented Jan 12, 2025 •

edited

Loading

tkrishtop commented Feb 25, 2025 •

edited

Loading