TELCOV10N-452 upd role junit2json post CR

- reduce patterns handline and calls to find
- reduce number of task
- update documentation

Signed-off-by: Maxim Kovgan <makovgan@redhat.com>

roles/junit2json/README.md

@@ -20,17 +20,16 @@ Description: Converts XML junit reports passed or in passed directory into singl
 #### Key: main 
 **Description**: The resulting JSON file(s) are of the same structure for all the teams' and CI systems and used later to be sent to the data collection system.
 This is the main entrypoint for the role `redhatci.ocp.junit2json`.
-Scans all the XML reports passed as `junit2json_input_reports_list`, scans folders for files matching `global_reports_path_patterns`.
-Converts XMLs into JSON, if variable `junit2json_do_merge` is `true` XMLs are merged into one.
-New filenames are based on the old ones and stored in `global_json_reports_list` global variable
+Converts XMLs into JSON, if variable `junit2json_do_merge` is `true`, multiple XMLs are merged into one XML file.
+New filename(s) is(are) based on the old ones and stored in global variable `global_json_reports_list`.
 
 
 
   - **junit2json_input_reports_list**
     - **Required**: True
     - **Type**: list
     - **Default**: none
-    - **Description**: Mixed List files (and/or directories) containing the XML report(s) to convert
+    - **Description**: List of JUnit XML report files to convert to JSON
 
 
 
@@ -113,12 +112,10 @@ it is generated under `junit2json_output_dir`
 | Generate content hash from the content | ansible.builtin.set_fact | False |
 | Print curr XML report filename | ansible.builtin.debug | True |
 | Obtain info on previous content checksum file | ansible.builtin.stat | False |
-| Update junit2json_result_data [`junit2json_do_merge: true`] | ansible.builtin.set_fact | True |
+| Update junit2json_result_data junit2json_do_merge=true | ansible.builtin.set_fact | True |
 | Convert junit XML to JSON and save in junit2json_result_data | ansible.builtin.set_fact | True |
-| Setup JSON report file name [`xml_report.endswith('.xml'): true`] | ansible.builtin.set_fact | True |
-| Setup JSON report file name [`xml_report.endswith('.xml'): false`] | ansible.builtin.set_fact | True |
-| Update junit2json_output_report_path [`junit2json_do_merge: true`] | ansible.builtin.set_fact | True |
-| Update junit2json_output_report_path [`junit2json_do_merge: false`] | ansible.builtin.set_fact | True |
+| Setup JSON report file name | ansible.builtin.set_fact | False |
+| Update junit2json_output_report_path junit2json_do_merge=true | ansible.builtin.set_fact | False |
 | Get xml_old_hash | ansible.builtin.set_fact | True |
 | Update global_json_reports_list | ansible.builtin.set_fact | False |
 | Ensure json data destination folder | ansible.builtin.file | True |
@@ -131,7 +128,7 @@ it is generated under `junit2json_output_dir`
 | ---- | ------ | --------- |
 | Generate XML files list | ansible.builtin.set_fact | False |
 | Print XML reports' filenames data | ansible.builtin.debug | True |
-| Merge multiple JUnit XML files into single consolidated report | ansible.builtin.shell | False |
+| Merge multiple JUnit XML files into single consolidated report | ansible.builtin.command | False |
 | Write merge resulting file | ansible.builtin.copy | True |
 | Override the xml files list for conversion | ansible.builtin.set_fact | False |
 
@@ -142,10 +139,7 @@ it is generated under `junit2json_output_dir`
 | Print path_item value | ansible.builtin.debug | False |
 | Setup helper variable | ansible.builtin.set_fact | False |
 | Check whether the path_item is a directory | ansible.builtin.stat | False |
-| Add path_item as a JUnit XML report [`stat.isdir: false`] | ansible.builtin.set_fact | True |
-| Find JUnit XML reports under path_item [`stat.isdir: true`] | ansible.builtin.find | True |
-| Setup the conversion list | ansible.builtin.set_fact | True |
-| Setup the default list for conversion | ansible.builtin.set_fact | True |
+| Update junit2json_reports_list with a JUnit XML report path_item | ansible.builtin.set_fact | True |
 
 #### File: tasks/main.yml
 
@@ -155,7 +149,7 @@ it is generated under `junit2json_output_dir`
 | Initialize input reports variable | ansible.builtin.debug | False |
 | Initialize reports variable | ansible.builtin.set_fact | False |
 | Expand the input list to list of existing files | ansible.builtin.include_tasks | False |
-| Merge JUnit XML reports into single file [`junit2json_do_merge: true`] | ansible.builtin.include_tasks | True |
+| Merge JUnit XML reports into single file junit2json_do_merge=true | ansible.builtin.include_tasks | True |
 | Convert XML to JSON | ansible.builtin.include_tasks | True |
 
 
@@ -181,18 +175,16 @@ classDef rescue stroke:#665352,stroke-width:2px;
   Read_file_content0-->|Task| Generate_content_hash_from_the_content1[generate content hash from the content]:::task
   Generate_content_hash_from_the_content1-->|Task| Print_curr_XML_report_filename2[print curr xml report filename<br>When: **junit2json debug**]:::task
   Print_curr_XML_report_filename2-->|Task| Obtain_info_on_previous_content_checksum_file3[obtain info on previous content checksum file]:::task
-  Obtain_info_on_previous_content_checksum_file3-->|Task| Update_junit2json_result_data___junit2json_do_merge__true__4[update junit2json result data   junit2json do<br>merge  true  <br>When: **junit2json do merge**]:::task
-  Update_junit2json_result_data___junit2json_do_merge__true__4-->|Task| Convert_junit_XML_to_JSON_and_save_in_junit2json_result_data5[convert junit xml to json and save in junit2json<br>result data<br>When: **not junit2json do merge**]:::task
-  Convert_junit_XML_to_JSON_and_save_in_junit2json_result_data5-->|Task| Setup_JSON_report_file_name___xml_report_endswith___xml____true__6[setup json report file name   xml report endswith <br> xml    true  <br>When: **not junit2json do merge  and xml report endswith  <br>xml**]:::task
-  Setup_JSON_report_file_name___xml_report_endswith___xml____true__6-->|Task| Setup_JSON_report_file_name___xml_report_endswith___xml____false__7[setup json report file name   xml report endswith <br> xml    false  <br>When: **not junit2json do merge  and not xml report<br>endswith   xml**]:::task
-  Setup_JSON_report_file_name___xml_report_endswith___xml____false__7-->|Task| Update_junit2json_output_report_path___junit2json_do_merge__true__8[update junit2json output report path   junit2json<br>do merge  true  <br>When: **junit2json do merge**]:::task
-  Update_junit2json_output_report_path___junit2json_do_merge__true__8-->|Task| Update_junit2json_output_report_path___junit2json_do_merge__false__9[update junit2json output report path   junit2json<br>do merge  false  <br>When: **not junit2json do merge**]:::task
-  Update_junit2json_output_report_path___junit2json_do_merge__false__9-->|Task| Get_xml_old_hash10[get xml old hash<br>When: **junit2json old hash stat stat exists**]:::task
-  Get_xml_old_hash10-->|Task| Update_global_json_reports_list11[update global json reports list]:::task
-  Update_global_json_reports_list11-->|Task| Ensure_json_data_destination_folder12[ensure json data destination folder<br>When: **junit2json xml report hash curr    junit2json xml<br>report hash old**]:::task
-  Ensure_json_data_destination_folder12-->|Task| Write_json_data_to_destination13[write json data to destination<br>When: **junit2json xml report hash curr    junit2json xml<br>report hash old**]:::task
-  Write_json_data_to_destination13-->|Task| Write_current_hash_to_destination14[write current hash to destination<br>When: **junit2json xml report hash curr    junit2json xml<br>report hash old**]:::task
-  Write_current_hash_to_destination14-->End
+  Obtain_info_on_previous_content_checksum_file3-->|Task| Update_junit2json_result_data_junit2json_do_merge_true4[update junit2json result data junit2json do merge<br>true<br>When: **junit2json do merge**]:::task
+  Update_junit2json_result_data_junit2json_do_merge_true4-->|Task| Convert_junit_XML_to_JSON_and_save_in_junit2json_result_data5[convert junit xml to json and save in junit2json<br>result data<br>When: **not junit2json do merge**]:::task
+  Convert_junit_XML_to_JSON_and_save_in_junit2json_result_data5-->|Task| Setup_JSON_report_file_name6[setup json report file name]:::task
+  Setup_JSON_report_file_name6-->|Task| Update_junit2json_output_report_path_junit2json_do_merge_true7[update junit2json output report path junit2json do<br>merge true]:::task
+  Update_junit2json_output_report_path_junit2json_do_merge_true7-->|Task| Get_xml_old_hash8[get xml old hash<br>When: **junit2json old hash stat stat exists**]:::task
+  Get_xml_old_hash8-->|Task| Update_global_json_reports_list9[update global json reports list]:::task
+  Update_global_json_reports_list9-->|Task| Ensure_json_data_destination_folder10[ensure json data destination folder<br>When: **junit2json xml report hash curr    junit2json xml<br>report hash old**]:::task
+  Ensure_json_data_destination_folder10-->|Task| Write_json_data_to_destination11[write json data to destination<br>When: **junit2json xml report hash curr    junit2json xml<br>report hash old**]:::task
+  Write_json_data_to_destination11-->|Task| Write_current_hash_to_destination12[write current hash to destination<br>When: **junit2json xml report hash curr    junit2json xml<br>report hash old**]:::task
+  Write_current_hash_to_destination12-->End
 ```
 
 
@@ -236,11 +228,8 @@ classDef rescue stroke:#665352,stroke-width:2px;
   Start-->|Task| Print_path_item_value0[print path item value]:::task
   Print_path_item_value0-->|Task| Setup_helper_variable1[setup helper variable]:::task
   Setup_helper_variable1-->|Task| Check_whether_the_path_item_is_a_directory2[check whether the path item is a directory]:::task
-  Check_whether_the_path_item_is_a_directory2-->|Task| Add_path_item_as_a_JUnit_XML_report___stat_isdir__false__3[add path item as a junit xml report   stat isdir <br>false  <br>When: **not  junit2json path item stat stat isdir  <br>default false**]:::task
-  Add_path_item_as_a_JUnit_XML_report___stat_isdir__false__3-->|Task| Find_JUnit_XML_reports_under_path_item___stat_isdir__true__4[find junit xml reports under path item   stat<br>isdir  true  <br>When: **junit2json path item stat stat isdir   default<br>false**]:::task
-  Find_JUnit_XML_reports_under_path_item___stat_isdir__true__4-->|Task| Setup_the_conversion_list5[setup the conversion list<br>When: **junit2json path item stat stat isdir   default<br>false  and  junit2json xml reports find matched  <br>0**]:::task
-  Setup_the_conversion_list5-->|Task| Setup_the_default_list_for_conversion6[setup the default list for conversion<br>When: **junit2json path item stat stat isdir   default<br>false  and junit2json found xml reports   length  <br>0**]:::task
-  Setup_the_default_list_for_conversion6-->End
+  Check_whether_the_path_item_is_a_directory2-->|Task| Update_junit2json_reports_list_with_a_JUnit_XML_report_path_item3[update junit2json reports list with a junit xml<br>report path item<br>When: **junit2json path item stat stat exists   default<br>false**]:::task
+  Update_junit2json_reports_list_with_a_JUnit_XML_report_path_item3-->End
 ```
 
 
@@ -262,7 +251,7 @@ classDef rescue stroke:#665352,stroke-width:2px;
   Validate_some_variables0-->|Task| Initialize_input_reports_variable1[initialize input reports variable]:::task
   Initialize_input_reports_variable1-->|Task| Initialize_reports_variable2[initialize reports variable]:::task
   Initialize_reports_variable2-->|Include task| expand_yml3[expand the input list to list of existing files<br>include_task: expand yml]:::includeTasks
-  expand_yml3-->|Include task| merge_yml4[merge junit xml reports into single file  <br>junit2json do merge  true  <br>When: **junit2json do merge and  junit2json found xml<br>reports matched   0**<br>include_task: merge yml]:::includeTasks
+  expand_yml3-->|Include task| merge_yml4[merge junit xml reports into single file<br>junit2json do merge true<br>When: **junit2json do merge and  junit2json found xml<br>reports matched   0**<br>include_task: merge yml]:::includeTasks
   merge_yml4-->|Include task| convert_yml5[convert xml to json<br>When: **junit2json reports list   length   0**<br>include_task: convert yml]:::includeTasks
   convert_yml5-->End
 ```
@@ -276,8 +265,6 @@ classDef rescue stroke:#665352,stroke-width:2px;
 - name: "Example playbook to use the role redhatci.ocp.junit2json role"
   hosts: localhost
   vars:
-    global_reports_path_patterns: # this depends on your actual naming patterns
-      - '*junit.xml'
     global_json_reports_list: []
     input_reports:
       - "/path/to/input1-junit.xml"

roles/junit2json/examples/example_playbook.yml

@@ -3,8 +3,6 @@
 - name: "Example playbook to use the role redhatci.ocp.junit2json role"
   hosts: localhost
   vars:
-    global_reports_path_patterns: # this depends on your actual naming patterns
-      - '*junit.xml'
     global_json_reports_list: []
     input_reports:
       - "/path/to/input1-junit.xml"

roles/junit2json/meta/argument_specs.yml

@@ -6,9 +6,8 @@ argument_specs:
     description: |
       The resulting JSON file(s) are of the same structure for all the teams' and CI systems and used later to be sent to the data collection system.
       This is the main entrypoint for the role `redhatci.ocp.junit2json`.
-      Scans all the XML reports passed as `junit2json_input_reports_list`, scans folders for files matching `global_reports_path_patterns`.
-      Converts XMLs into JSON, if variable `junit2json_do_merge` is `true` XMLs are merged into one.
-      New filenames are based on the old ones and stored in `global_json_reports_list` global variable
+      Converts XMLs into JSON, if variable `junit2json_do_merge` is `true`, multiple XMLs are merged into one XML file.
+      New filename(s) is(are) based on the old ones and stored in global variable `global_json_reports_list`.
     author:
       - Max Kovgan
     options:
@@ -17,7 +16,7 @@ argument_specs:
         required: true
         elements: str
         description: |
-          Mixed List files (and/or directories) containing the XML report(s) to convert
+          List of JUnit XML report files to convert to JSON
       junit2json_output_dir:
         type: str
         required: true

roles/junit2json/tasks/convert.yml

@@ -5,19 +5,19 @@
 
 - name: Read file content
   ansible.builtin.set_fact:
-    junit2json_xml_report_content: "{{ lookup('ansible.builtin.file', xml_report) }}"
+    junit2json_xml_report_content: "{{ lookup('ansible.builtin.file', item) }}"
 
 - name: Generate content hash from the content
   ansible.builtin.set_fact:
-    junit2json_xml_report_hash_path: "{{ xml_report }}.{{ junit2json_hash }}"
+    junit2json_xml_report_hash_path: "{{ item }}.{{ junit2json_hash }}"
     junit2json_xml_report_hash_curr: "{{ junit2json_xml_report_content | ansible.builtin.hash(junit2json_hash) }}"
     junit2json_xml_report_hash_old: ""
 
 # debug only:
 - name: Print curr XML report filename
   ansible.builtin.debug:
     msg:
-      - "XML report path is: {{ xml_report }}"
+      - "XML report path is: {{ item }}"
   when:
     - junit2json_debug
 
@@ -27,7 +27,7 @@
     checksum_algorithm: "{{ junit2json_hash }}"
   register: _junit2json_old_hash_stat
 
-- name: "Update junit2json_result_data [`junit2json_do_merge: true`]"
+- name: Update junit2json_result_data junit2json_do_merge=true
   ansible.builtin.set_fact:
     junit2json_result_data: "{{ _junit2json_merged_xml.stdout | redhatci.ocp.junit2obj | to_nice_json }}"
   when:
@@ -39,31 +39,13 @@
   when:
     - not(junit2json_do_merge)
 
-- name: "Setup JSON report file name [`xml_report.endswith('.xml'): true`]"
+- name: Setup JSON report file name
   ansible.builtin.set_fact:
-    junit2json_json_report_path: "{{ xml_report | basename | regex_replace('\\.xml$', '.json') }}"
-  when:
-    - not(junit2json_do_merge)
-    - xml_report.endswith(".xml")
-
-- name: "Setup JSON report file name [`xml_report.endswith('.xml'): false`]"
-  ansible.builtin.set_fact:
-    junit2json_json_report_path: "{{ xml_report | basename }}.json"
-  when:
-    - not(junit2json_do_merge)
-    - not(xml_report.endswith(".xml"))
+    junit2json_json_report_path: "{{ item | basename | regex_replace('\\.xml$', '.json') if item.endswith('.xml') else item | basename + '.json' }}"
 
-- name: "Update junit2json_output_report_path [`junit2json_do_merge: true`]"
+- name: Update junit2json_output_report_path junit2json_do_merge=true
   ansible.builtin.set_fact:
-    junit2json_output_report_path: "{{ junit2json_output_dir }}/{{ junit2json_output_merged_report }}"
-  when:
-    - junit2json_do_merge
-
-- name: "Update junit2json_output_report_path [`junit2json_do_merge: false`]"
-  ansible.builtin.set_fact:
-    junit2json_output_report_path: "{{ junit2json_output_dir }}/{{ junit2json_json_report_path }}"
-  when:
-    - not(junit2json_do_merge)
+    junit2json_output_report_path: "{{ junit2json_output_dir }}/{{ junit2json_output_merged_report if junit2json_do_merge else junit2json_json_report_path }}"
 
 - name: Get xml_old_hash
   ansible.builtin.set_fact:

roles/junit2json/tasks/expand.yml

@@ -1,47 +1,19 @@
 # SPDX-License-Identifier: Apache-2.0
 ---
 # tasks file for role `redhatci.ocp.junit2json`
-# collects possibly multiple XMLs from `path_item` (when it is a directory)
+# collects possibly multiple XMLs from `item` (when it is a directory)
 
 - name: Print path_item value
   ansible.builtin.debug:
-    msg: "path_item: {{ path_item }}"
+    msg: "path_item: {{ item }}"
 
-- name: Setup helper variable
-  ansible.builtin.set_fact:
-    junit2json_find_patterns: "{{ global_reports_path_patterns | map('regex_replace', '^', '*.') | list }}"
-    cacheable: false
-
-- name: Check whether the path_item is a directory
+- name: Collect the item stat
   ansible.builtin.stat:
-    path: "{{ path_item }}"
+    path: "{{ item }}"
   register: _junit2json_path_item_stat
 
-- name: "Add path_item as a JUnit XML report [`stat.isdir: false`]"
-  ansible.builtin.set_fact:
-    junit2json_reports_list: "{{ junit2json_reports_list + [path_item] }}"
-  when:
-    - not(_junit2json_path_item_stat.stat.isdir | default(false))
-
-- name: "Find JUnit XML reports under path_item [`stat.isdir: true`]"
-  ansible.builtin.find:
-    paths: "{{ path_item }}"
-    patterns: "{{ global_reports_path_patterns }}"
-    recurse: false
-  register: _junit2json_xml_reports_find
-  when:
-    - _junit2json_path_item_stat.stat.isdir | default(false)
-
-- name: Setup the conversion list
-  ansible.builtin.set_fact:
-    junit2json_found_xml_reports: "{{ _junit2json_xml_reports_find.files | map(attribute='path') | list }}"
-  when:
-    - _junit2json_path_item_stat.stat.isdir | default(false)
-    - _junit2json_xml_reports_find.matched > 0
-
-- name: Setup the default list for conversion
+- name: Update junit2json_reports_list with a JUnit XML report item
   ansible.builtin.set_fact:
-    junit2json_reports_list: "{{ junit2json_reports_list + junit2json_found_xml_reports }}"
+    junit2json_reports_list: "{{ junit2json_reports_list + [item] }}"
   when:
-    - _junit2json_path_item_stat.stat.isdir | default(false)
-    - junit2json_found_xml_reports | length > 0
+    - _junit2json_path_item_stat.stat.exists | default(false)

roles/junit2json/tasks/main.yml

@@ -4,13 +4,10 @@
 
 - name: Validate some variables
   ansible.builtin.assert:
-    that: "{{ condition }}"
-  loop:
-    - global_reports_path_patterns | length > 0
-    - junit2json_input_reports_list | length > 0
-    - junit2json_output_dir | length > 0
-  loop_control:
-    loop_var: condition
+    that:
+      - junit2json_input_reports_list | length > 0
+      - junit2json_output_dir | length > 0
+    fail_msg: "One of the mandatory variables is missing or empty"
 
 - name: Initialize input reports variable
   ansible.builtin.debug:
@@ -24,10 +21,8 @@
   ansible.builtin.include_tasks:
     file: expand.yml
   loop: "{{ junit2json_input_reports_list }}"
-  loop_control:
-    loop_var: path_item
 
-- name: "Merge JUnit XML reports into single file [`junit2json_do_merge: true`]"
+- name: Merge JUnit XML reports into single file junit2json_do_merge=true
   ansible.builtin.include_tasks:
     file: merge.yml
   when:
@@ -38,7 +33,5 @@
   ansible.builtin.include_tasks:
     file: convert.yml
   loop: "{{ junit2json_reports_list }}"
-  loop_control:
-    loop_var: xml_report
   when:
-    - junit2json_reports_list | length > 0
+    - junit2json_reports_list | default([]) | length > 0