Overhaul python configuration of EDM conversion tools

doc/starterkit/k4MarlinWrapperCLIC/CEDViaWrapper.md

@@ -83,7 +83,8 @@ MyCEDViewer.Parameters = {
 
 # EDM4hep to LCIO converter
 edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")
-edmConvTool.Parameters = ["*"]
+edmConvTool.convertAll = True
+edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'}
 edmConvTool.OutputLevel = DEBUG
 MyCEDViewer.EDM4hep2LcioTool = edmConvTool
 
@@ -117,16 +118,4 @@ The `event_display.py` options file that is used above and that is present in th
 - Exchange the LCIO input reading by the podio input reading (see above)
 - Attach the `EDM4hep2LcioTool` to the wrapped `CEDViewer` processor
 
-This should allow to arrive at a similar steering file even for slightly different configurations. One potential pitfal is the slightly different naming of the `ddsim` outputs between LCIO and EDM4hep (see [this issue](https://github.com/AIDASoft/DD4hep/issues/921)). For the `event_display.py` in the `examples` drawing the `MCParticle`s in the event display required to change the name of the collection that is used in the `DrawInLayers` configuration parameter of the `CEDViewer`:
-
-```diff
-@@ -156,7 +156,7 @@ MyCEDViewer.Parameters = {
-                               "MarlinTrkTracks", "0", "6", "7",
-                               "PandoraClusters", "0", "3", "8",
-                               "PandoraPFOs", "0", "3", "9",
--                              "MCParticle", "0", "3", "0",
-+                              "MCParticles", "0", "3", "0",
-                               "VertexBarrelHits", "0", "5", "11",
-                               "VertexEndcapHits", "0", "5", "11",
-                               "InnerTrackerBarrelHits", "0", "5", "11",
-```
+This should allow one to arrive at a similar steering file even for slightly different configurations. One potential pitfall is the slightly different naming of the `ddsim` outputs between LCIO and EDM4hep (see [this issue](https://github.com/AIDASoft/DD4hep/issues/921)). This can be addressed by configuring the EDM4hep to LCIO converter (`edmConvTool` in the code above) to map the names accordingly.

doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md

@@ -117,17 +117,18 @@ Note and review the following when using the converters:
 - If using the `PodioInput` to read events: collection names must be indicated both in the `PodioInput` and the `EDM4hep2LcioTool`.
 - Collections not indicated to be converted **will not** be converted even if its a dependency from an indicated collection to be converted.
 - If a converted collection is used later by a Gaudi Algorithm, and this Gaudi Algorithm indicates the use of that collection in the `Parameters`, the converted collection name must match the name indicated in the Gaudi Algorithm `Parameters`.
-  + For example: A collection may be converted with the following parameters: `ReconstructedParticles", "ReconstructedParticleLCIO"`
+  + For example: A collection may be converted with the following parameters: `"ReconstructedParticles": "ReconstructedParticleLCIO"`
   + A Gaudi Algorithm may indicate in their `Parameters`: `"PFOCollection": ["ReconstructedParticleLCIO"]`
 
 ###  EDM4hep to LCIO converter
 
 Collections from events that are already read, or are produced by a Gaudi Algorithm can be converted from EDM4hep to LCIO format:
 
-1. Instantiate the `EDM4hep2LcioTool` Gaudi Tool.
-2. Indicate the collections to convert in `Parameters`.
-  + To convert all available collections write an asterisk, like follows: `edmConvTool.Parameters = ["*"]`
-  + Arguments are read in groups of 2: name of the EDM4hep collection, name of the LCIO converted collection.
+1. Instantiate the `EDM4hep2LcioTool` Gaudi Tool, e.g. as `edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")`
+2. Indicate the collections to convert using the options of the tool.
+  + To simply convert all available collections use `edmConvTool.convertAll = True` (this is also the default!)
+  + If you want to convert all available collections but want to rename some of them, e.g. because an algorithm expects a different name, use the `collNameMapping` option. This maps the input name to the output name; `edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'}` will convert the input `'MCParticles'` into the `'MCParticle'` collection.
+  + To convert only a subset of all available collections, set the `convertAll` option to `False` and indicate the collections to convert via the `collNameMapping` option.
 3. Select the Gaudi Algorithm that will convert the indicated collections.
 4. Add the Tool to the Gaudi Algorithm.
 
@@ -137,10 +138,11 @@ from Configurables import ToolSvc, EDM4hep2LcioTool
 # 1
 edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")
 # 2
-edmConvTool.Parameters = [
-    "EFlowTrack", "EFlowTrack_LCIO",
-    "ReconstructedParticles", "ReconstructedParticle_LCIO"
-]
+edmConvToll.convertAll = False
+edmConvTool.collNameMapping = {
+    "EFlowTrack": "EFlowTrack_LCIO",
+    "ReconstructedParticles": "ReconstructedParticle_LCIO"
+}
 edmConvTool.OutputLevel = DEBUG
 
 # 3
@@ -154,10 +156,11 @@ InitDD4hep.EDM4hep2LcioTool=edmConvTool
 
 Collections from events that are already read, or are produced by a gaudi Algorithm can be converted from LCIO to EDM4hep format:
 
-1. Instantiate the `Lcio2EDM4hepTool` Gaudi Tool.
-2. Indicate the collections to convert in `Parameters`.
-  + To convert all available collections write an asterisk, like follows: `lcioConvTool.Parameters = ["*"]`
-  + Arguments are read in groups of 2: name of the LCIO collection, name of the EDM4hep converted collection.
+1. Instantiate the `Lcio2EDM4hepTool` Gaudi Tool, e.g. as `lcioConvTool = Lcio2EDM4hepTool("LCIO2EDM4hep")`.
+2. Indicate the collections to convert in the options of the tool.
+  + To simply convert all available collections use `lcioConvTool.convertAll = True` (this is also the default!)
+  + If you want to convert all available collections but want to rename some of them, e.g. because an algorithm expects a different name, use the `collNameMapping` option. This maps the input name to the output name; `lcioConvTool.collNameMapping = {'MCParticles': 'MCParticle'}` will convert the input `'MCParticles'` into the `'MCParticle'` collection.
+  + To convert only a subset of all available collections, set the `convertAll` option to `False` and indicate the collections to convert via the `collNameMapping` option.
 3. Select the Gaudi Algorithm that will convert the indicated collections.
 4. Add the Tool to the Gaudi Algorithm.
 
@@ -167,12 +170,13 @@ from Configurables import ToolSvc, Lcio2EDM4hepTool
 # 1
 lcioConvTool = Lcio2EDM4hepTool("LCIO2EDM4hep")
 # 2
-lcioConvTool.Parameters = [
-    "EFlowTrackConv", "EFlowTrackEDM4hep",
-    "ReconstructedParticle", "ReconstructedParticlesEDM4hep",
-    "BuildUpVertices", "BuildUpVerticesEDM4hep",
-    "PrimaryVertices", "PrimaryVerticesEDM4hep"
-]
+lcioConvTool.convertAll = False
+lcioConvTool.collNameMapping = {
+    "EFlowTrackConv": "EFlowTrackEDM4hep",
+    "ReconstructedParticle": "ReconstructedParticlesEDM4hep",
+    "BuildUpVertices": "BuildUpVerticesEDM4hep",
+    "PrimaryVertices": "PrimaryVerticesEDM4hep"
+}
 lcioConvTool.OutputLevel = DEBUG
 
 # 3
@@ -181,5 +185,3 @@ JetClusteringAndRefiner = MarlinProcessorWrapper("JetClusteringAndRefiner")
 # 4
 JetClusteringAndRefiner.Lcio2EDM4hepTool=lcioConvTool
 ```
-
-

k4MarlinWrapper/examples/event_display.py

@@ -156,7 +156,7 @@
                               "MarlinTrkTracks", "0", "6", "7",
                               "PandoraClusters", "0", "3", "8",
                               "PandoraPFOs", "0", "3", "9",
-                              "MCParticles", "0", "3", "0",
+                              "MCParticle", "0", "3", "0",
                               "VertexBarrelHits", "0", "5", "11",
                               "VertexEndcapHits", "0", "5", "11",
                               "InnerTrackerBarrelHits", "0", "5", "11",
@@ -193,7 +193,8 @@
 
 # EDM4hep to LCIO converter
 edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")
-edmConvTool.Parameters = ["*"]
+edmConvTool.convertAll = True
+edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'}
 edmConvTool.OutputLevel = DEBUG
 MyCEDViewer.EDM4hep2LcioTool = edmConvTool
 

k4MarlinWrapper/k4MarlinWrapper/converters/EDM4hep2Lcio.h

@@ -1,25 +1,26 @@
 #ifndef K4MARLINWRAPPER_EDM4HEP2LCIO_H
 #define K4MARLINWRAPPER_EDM4HEP2LCIO_H
 
-// std
-#include <algorithm>
-#include <bitset>
-#include <string>
-#include <vector>
-
-// GAUDI
-#include <GaudiAlg/GaudiTool.h>
+// k4MarlinWrapper
+#include "k4MarlinWrapper/LCEventWrapper.h"
+#include "k4MarlinWrapper/converters/IEDMConverter.h"
+#include "k4MarlinWrapper/util/k4MarlinWrapperUtil.h"
 
 // FWCore
 #include <k4FWCore/DataHandle.h>
 
 //k4EDM4hep2LcioConv
 #include "k4EDM4hep2LcioConv/k4EDM4hep2LcioConv.h"
 
-// k4MarlinWrapper
-#include "k4MarlinWrapper/LCEventWrapper.h"
-#include "k4MarlinWrapper/converters/IEDMConverter.h"
-#include "k4MarlinWrapper/util/k4MarlinWrapperUtil.h"
+// GAUDI
+#include <GaudiAlg/GaudiTool.h>
+
+// std
+#include <algorithm>
+#include <bitset>
+#include <map>
+#include <string>
+#include <vector>
 
 class EDM4hep2LcioTool : public GaudiTool, virtual public IEDMConverter {
 public:
@@ -31,7 +32,8 @@ class EDM4hep2LcioTool : public GaudiTool, virtual public IEDMConverter {
   StatusCode convertCollections(lcio::LCEventImpl* lcio_event);
 
 private:
-  Gaudi::Property<std::vector<std::string>> m_edm2lcio_params{this, "Parameters", {}};
+  Gaudi::Property<std::map<std::string, std::string>> m_collNames{this, "collNameMapping", {}};
+  Gaudi::Property<bool>                               m_convertAll{this, "convertAll", true};
 
   PodioDataSvc*                   m_podioDataSvc;
   ServiceHandle<IDataProviderSvc> m_eventDataSvc;

k4MarlinWrapper/k4MarlinWrapper/converters/Lcio2EDM4hep.h

@@ -15,6 +15,9 @@
 // Converter Interface
 #include "k4MarlinWrapper/converters/IEDMConverter.h"
 
+#include <map>
+#include <string>
+
 class Lcio2EDM4hepTool : public GaudiTool, virtual public IEDMConverter {
 public:
   Lcio2EDM4hepTool(const std::string& type, const std::string& name, const IInterface* parent);
@@ -25,7 +28,8 @@ class Lcio2EDM4hepTool : public GaudiTool, virtual public IEDMConverter {
   StatusCode convertCollections(lcio::LCEventImpl* lcio_event);
 
 private:
-  Gaudi::Property<std::vector<std::string>> m_params{this, "Parameters", {}};
+  Gaudi::Property<std::map<std::string, std::string>> m_collNames{this, "collNameMapping", {}};
+  Gaudi::Property<bool>                               m_convertAll{this, "convertAll", true};
 
   std::map<std::string, DataObjectHandleBase*> m_dataHandlesMap;
 

k4MarlinWrapper/src/components/EDM4hep2Lcio.cpp

@@ -1,4 +1,5 @@
 #include "k4MarlinWrapper/converters/EDM4hep2Lcio.h"
+#include <unordered_map>
 
 DECLARE_COMPONENT(EDM4hep2LcioTool);
 
@@ -270,34 +271,30 @@ void EDM4hep2LcioTool::convertAdd(const std::string& e4h_coll_name, const std::s
 // Parse property parameters and convert the indicated collections.
 // Use the collection names in the parameters to read and write them
 StatusCode EDM4hep2LcioTool::convertCollections(lcio::LCEventImpl* lcio_event) {
-  CollectionsPairVectors collection_pairs{};
+  const auto collections = m_podioDataSvc->getCollections();
 
-  // Convert collections from parameters
-  if (m_edm2lcio_params.size() > 1 && m_edm2lcio_params.size() % 2 == 0) {
-    for (int i = 0; i < m_edm2lcio_params.size(); i = i + 2) {
-      if (!collectionExist(m_edm2lcio_params[i + 1], lcio_event)) {
-        convertAdd(m_edm2lcio_params[i], m_edm2lcio_params[i + 1], lcio_event, collection_pairs);
-      } else {
-        debug() << " Collection " << m_edm2lcio_params[i + 1] << " already in place, skipping conversion. " << endmsg;
-      }
+  // Start off with the pre-defined collection name mappings
+  auto collsToConvert{m_collNames.value()};
+  if (m_convertAll) {
+    info() << "Converting all collections from EDM4hep to LCIO" << endmsg;
+    // And simply add the rest, exploiting the fact that emplace will not
+    // replace existing entries with the same key
+    for (const auto& [name, _] : collections) {
+      collsToConvert.emplace(name, name);
     }
-
-    FillMissingCollections(collection_pairs);
   }
 
-  // Convert all collections if the only parameter is "*"
-  else if (m_edm2lcio_params.size() == 1 && m_edm2lcio_params[0] == "*") {
-    info() << "Converting all collections from EDM4hep to LCIO" << endmsg;
+  CollectionsPairVectors collection_pairs{};
+  for (const auto& [edm4hepName, lcioName] : collsToConvert) {
+    if (!collectionExist(lcioName, lcio_event)) {
+      convertAdd(edm4hepName, lcioName, lcio_event, collection_pairs);
+    } else {
+      debug() << " Collection " << lcioName << " already in place, skipping conversion. " << endmsg;
+    }
 
-    const auto& collections = m_podioDataSvc->getCollections();
-    for (auto& [name, collection] : collections) {
-      convertAdd(name, name, lcio_event, collection_pairs);
+    if (!m_convertAll) {
+      FillMissingCollections(collection_pairs);
     }
-  } else {
-    error() << " Error processing conversion parameters. 2 arguments (EDM4hep "
-               "name, LCIO name) per collection, or 1 argument \"*\" to convert all collections expected. "
-            << endmsg;
-    return StatusCode::FAILURE;
   }
 
   return StatusCode::SUCCESS;