Allow ParameterExpression values in TemplateOptimization pass (#6899

)

* added new equivalence rzx_zz4

* added another zz-ish template (looks for basis gates)

* looking at adding additions to rzx templates

* fixes parsing of Parameter name for sympy during template optimization

* generalized fix for multiple parameters in an expression and constant parameters, removed some excess files from another branch

* one more minor fix to bring inline with qiskit-terra main

* trying to fix parameter parsing for template optimization

* made sure floats were added correctly to (template) parameters

* got template matching to work with floats, sympy still not understanding Parameters either in equations or symbols set in template_substitution.py

* further modifications to accept circuit parameters for template optimization

* debugging the binding of template parameters

* realized I was not treating ParameterExpressions carefully and should lean on parse_expr from sympy to cast to symbols

* converted all bindings to ParameterExpressions (via .assign()), set trace to fix template use with Parameters

* cleaned up _attempt_bind routine

* exploring overriding removing the matched scenarios with parameters

* introduced a total hack for dealing with ParamterExpressions that contain floats for RXGates

* (non-latex) parameters now currently working in template optimization transpilig step followed by parameter binding

* cleaned up some whitespace and removed commented-out lines

* cleaned up some tox/lint errors

* removed unneccessary Parameter import

* bypassed unit test test_unbound_parameters() and re-tox/lint

* fixed one last linting issue

* fixed cyclic import error

* modified calibration_creator to accept a ParameterExpression containing a float

* fixed an mismatch when trying to add calibrations and addressed a conversation in the PR

* last tox/lint checks i'm sure ;)

* now params comes from node_op argument

* handling error in case gate parameters is empty in dagcircuit.py

* Fix template matching for parameters with LaTeX name.

* added missing docstring

* removed pdb set_trace

* made changes requested in PR 6899

* made changes requested in PR 6899 #2

* remembered to tighten try/except handling

* finished making changes requested in PR 6899

* fixed remaining linting issue

* added comment about templates working for parameterized RZXGates and RXGates

* Fix test unbound parameters

* Check if matrix with Parameter is unitary

* Fix merge issue

* removed real=True in two symbol Symbol expressions, which was messing up the solver for some reason.

* generalized to iterate over parameters, and removed reference to private member

* modified .get_sympy_expr() to use symengine if possible

* made the negation of the RXGate() much less verbose

* working thru tox/lint checks

* added unit test test_unbound_parameters_in_rzx_templates to confirm template optimization handles template optimization with unbound parameters correctly

* Fix unbund parameters test

* fixed issue with adding calibrations without params

* Add real=True to symbols

* fixed linting issue

* Fix for symengine

* simplified the parameter handling for adding calibrations to gates

* added a check for unitary on an arbitrary float in the case symengine is imported (and fixed a couple minor bugs)

* Parammeter can be complex

* fixed tox/lint issues

* removed one more imposition of real parameters

* one last linting issue

* modified release notes

* fixed some transpiler library imports that were out of date

* added sphinx referencing to release notes and print statement for the case of testing unbound parameters when creating a unitary gate

* fixed some tox/lint issues

* Fix review issues

* fixing last tox/lint issues

* added release notes and fixed tox/lint issues

* added method in template_substitution to compare the number of parameters in circuit with the that of the template that would potentially replace it

* fixing up some template matching unit tests

* fixed up template matching unit tests to remove calls to UnitaryGate

* Update qiskit/dagcircuit/dagcircuit.py

Co-authored-by: Jake Lishman <jake@binhbar.com>

* Update qiskit/extensions/unitary.py

Co-authored-by: Jake Lishman <jake@binhbar.com>

* Update qiskit/extensions/unitary.py

Co-authored-by: Jake Lishman <jake@binhbar.com>

* added template test with two parameters and new logic in the case there are duplicate parameters in the circuit and template

* added two-parameter unit test and a check for overlapping parameters between circuits and templates

* remove ParameterTypeeException from exceptions.py

* Restore lazy symengine imports

Use of `_optionals.HAS_SYMENGINE` is intended to be within run-time
locations; putting it at the top level of a file causes this fairly
heavy library to be imported at runtime, slowing down `import qiskit`
for those who won't use that functionality.

* Rename to_sympify_expression to sympify

* Revert now-unnecessary changes to calibration builder

* Fixup release note

* Add explicit tests of template matching pass

This adds several tests of the exact form produced by running the
template-matching transpiler pass, including those with purely numeric
circuits and those with symbolic parameters.

* Fix template parameter substitution

This fixes issues introduced recently in the PR that caused parameters
to be incorrectly bound in the result.  This meant that the actual
numbers in the produced circuits were incorrect.  This happened mostly
by tracking data structures being updated at the wrong levels within
loops.

In addition, this commit also updates some data structures to more
robust and efficient versions:

- Testing whether a parameter has a clash is best done by constructing a
  set of parameters used in the input circuits, then testing directly on
  this, rather than stringifying expressions and using subsearch
  matches; this avoids problems if two parameters have contained names,
  or if more than one match is catenated into a single string.

- Using a dictionary with a missing-element constructor to build the
  replacement parameters allows the looping logic to be simpler; the
  "build missing element" logic can be separated out to happen
  automatically.

* Fix overlooked documentation comment

* Remove qasm.pi import in favour of numpy

* Add tests of multi-parameter instructions

* Fix template matching with multiparameter expressions

* Silence overzealous pylint warning

Co-authored-by: Rafał Pracht <rpracht@wp.pl>
Co-authored-by: Rafał Pracht <55279376+rafal-pracht@users.noreply.github.com>
Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

qiskit/circuit/parameterexpression.py

@@ -22,7 +22,6 @@
 from qiskit.circuit.exceptions import CircuitError
 from qiskit.utils import optionals as _optionals
 
-
 # This type is redefined at the bottom to insert the full reference to "ParameterExpression", so it
 # can safely be used by runtime type-checkers like Sphinx.  Mypy does not need this because it
 # handles the references by static analysis.
@@ -522,6 +521,20 @@ def is_real(self):
                 return False
         return True
 
+    def sympify(self):
+        """Return symbolic expression as a raw Sympy or Symengine object.
+
+        Symengine is used preferentially; if both are available, the result will always be a
+        ``symengine`` object.  Symengine is a separate library but has integration with Sympy.
+
+        .. note::
+
+            This is for interoperability only.  Qiskit will not accept or work with raw Sympy or
+            Symegine expressions in its parameters, because they do not contain the tracking
+            information used in circuit-parameter binding and assignment.
+        """
+        return self._symbol_expr
+
 
 # Redefine the type so external imports get an evaluated reference; Sphinx needs this to understand
 # the type hints.

qiskit/transpiler/passes/optimization/template_matching/template_substitution.py

@@ -14,9 +14,11 @@
 Template matching substitution, given a list of maximal matches it substitutes
 them in circuit and creates a new optimized dag version of the circuit.
 """
+import collections
 import copy
+import itertools
 
-from qiskit.circuit import ParameterExpression
+from qiskit.circuit import Parameter, ParameterExpression
 from qiskit.dagcircuit.dagcircuit import DAGCircuit
 from qiskit.dagcircuit.dagdependency import DAGDependency
 from qiskit.converters.dagdependency_to_dag import dagdependency_to_dag
@@ -175,7 +177,6 @@ def _rules(self, circuit_sublist, template_sublist, template_complement):
         Returns:
             bool: True if the match respects the given rule for replacement, False otherwise.
         """
-
         if self._quantum_cost(template_sublist, template_complement):
             for elem in circuit_sublist:
                 for config in self.substitution_list:
@@ -269,11 +270,6 @@ def _remove_impossible(self):
         list_predecessors = []
         remove_list = []
 
-        # First remove any scenarios that have parameters in the template.
-        for scenario in self.substitution_list:
-            if scenario.has_parameters():
-                remove_list.append(scenario)
-
         # Initialize predecessors for each group of matches.
         for scenario in self.substitution_list:
             predecessors = set()
@@ -324,8 +320,7 @@ def _substitution(self):
 
             # Fake bind any parameters in the template
             template = self._attempt_bind(template_sublist, circuit_sublist)
-
-            if template is None:
+            if template is None or self._incr_num_parameters(template):
                 continue
 
             template_list = range(0, self.template_dag_dep.size())
@@ -432,7 +427,6 @@ def run_dag_opt(self):
                         cargs = []
                     node = group.template_dag_dep.get_node(index)
                     inst = node.op.copy()
-
                     dag_dep_opt.add_op_node(inst.inverse(), qargs, cargs)
 
             # Add the unmatched gates.
@@ -486,6 +480,11 @@ def _attempt_bind(self, template_sublist, circuit_sublist):
         solution is found then the match is valid and the parameters
         are assigned. If not, None is returned.
 
+        In order to resolve the conflict of the same parameter names in the
+        circuit and template, each variable in the template sublist is
+        re-assigned to a new dummy parameter with a completely separate name
+        if it clashes with one that exists in an input circuit.
+
         Args:
             template_sublist (list): part of the matched template.
             circuit_sublist (list): part of the matched circuit.
@@ -499,51 +498,127 @@ def _attempt_bind(self, template_sublist, circuit_sublist):
         from sympy.parsing.sympy_parser import parse_expr
 
         circuit_params, template_params = [], []
+        # Set of all parameter names that are present in the circuits to be optimised.
+        circuit_params_set = set()
 
         template_dag_dep = copy.deepcopy(self.template_dag_dep)
 
-        for idx, t_idx in enumerate(template_sublist):
+        # add parameters from circuit to circuit_params
+        for idx, _ in enumerate(template_sublist):
             qc_idx = circuit_sublist[idx]
-            circuit_params += self.circuit_dag_dep.get_node(qc_idx).op.params
-            template_params += template_dag_dep.get_node(t_idx).op.params
+            parameters = self.circuit_dag_dep.get_node(qc_idx).op.params
+            circuit_params += parameters
+            for parameter in parameters:
+                if isinstance(parameter, ParameterExpression):
+                    circuit_params_set.update(x.name for x in parameter.parameters)
+
+        _dummy_counter = itertools.count()
+
+        def dummy_parameter():
+            # Strictly not _guaranteed_ to avoid naming clashes, but if someone's calling their
+            # parameters this then that's their own fault.
+            return Parameter(f"_qiskit_template_dummy_{next(_dummy_counter)}")
+
+        # Substitutions for parameters that have clashing names between the input circuits and the
+        # defined templates.
+        template_clash_substitutions = collections.defaultdict(dummy_parameter)
+
+        # add parameters from template to template_params, replacing parameters with names that
+        # clash with those in the circuit.
+        for t_idx in template_sublist:
+            node = template_dag_dep.get_node(t_idx)
+            sub_node_params = []
+            for t_param_exp in node.op.params:
+                if isinstance(t_param_exp, ParameterExpression):
+                    for t_param in t_param_exp.parameters:
+                        if t_param.name in circuit_params_set:
+                            new_param = template_clash_substitutions[t_param.name]
+                            t_param_exp = t_param_exp.assign(t_param, new_param)
+                sub_node_params.append(t_param_exp)
+                template_params.append(t_param_exp)
+            node.op.params = sub_node_params
+
+        for node in template_dag_dep.get_nodes():
+            sub_node_params = []
+            for param_exp in node.op.params:
+                if isinstance(param_exp, ParameterExpression):
+                    for param in param_exp.parameters:
+                        if param.name in template_clash_substitutions:
+                            param_exp = param_exp.assign(
+                                param, template_clash_substitutions[param.name]
+                            )
+                sub_node_params.append(param_exp)
+
+            node.op.params = sub_node_params
 
         # Create the fake binding dict and check
-        equations, symbols, sol, fake_bind = [], set(), {}, {}
-        for t_idx, params in enumerate(template_params):
-            if isinstance(params, ParameterExpression):
-                equations.append(sym.Eq(parse_expr(str(params)), circuit_params[t_idx]))
-                for param in params.parameters:
-                    symbols.add(param)
-
-        if not symbols:
+        equations, circ_dict, temp_symbols, sol, fake_bind = [], {}, {}, {}, {}
+        for circuit_param, template_param in zip(circuit_params, template_params):
+            if isinstance(template_param, ParameterExpression):
+                if isinstance(circuit_param, ParameterExpression):
+                    circ_param_sym = circuit_param.sympify()
+                else:
+                    circ_param_sym = parse_expr(str(circuit_param))
+                equations.append(sym.Eq(template_param.sympify(), circ_param_sym))
+
+                for param in template_param.parameters:
+                    temp_symbols[param] = param.sympify()
+
+                if isinstance(circuit_param, ParameterExpression):
+                    for param in circuit_param.parameters:
+                        circ_dict[param] = param.sympify()
+            elif template_param != circuit_param:
+                # Both are numeric parameters, but aren't equal.
+                return None
+
+        if not temp_symbols:
             return template_dag_dep
 
         # Check compatibility by solving the resulting equation
-        sym_sol = sym.solve(equations)
+        sym_sol = sym.solve(equations, set(temp_symbols.values()))
         for key in sym_sol:
             try:
-                sol[str(key)] = float(sym_sol[key])
+                sol[str(key)] = ParameterExpression(circ_dict, sym_sol[key])
             except TypeError:
                 return None
 
         if not sol:
             return None
 
-        for param in symbols:
-            fake_bind[param] = sol[str(param)]
+        for key in temp_symbols:
+            fake_bind[key] = sol[str(key)]
 
         for node in template_dag_dep.get_nodes():
             bound_params = []
-
-            for param in node.op.params:
-                if isinstance(param, ParameterExpression):
-                    try:
-                        bound_params.append(float(param.bind(fake_bind)))
-                    except KeyError:
-                        return None
+            for param_exp in node.op.params:
+                if isinstance(param_exp, ParameterExpression):
+                    for param in param_exp.parameters:
+                        if param in fake_bind:
+                            if fake_bind[param] not in bound_params:
+                                param_exp = param_exp.assign(param, fake_bind[param])
                 else:
-                    bound_params.append(param)
+                    param_exp = float(param_exp)
+                bound_params.append(param_exp)
 
             node.op.params = bound_params
 
         return template_dag_dep
+
+    def _incr_num_parameters(self, template):
+        """
+        Checks if template substitution would increase the number of
+        parameters in the circuit.
+        """
+        template_params = set()
+        for param_list in (node.op.params for node in template.get_nodes()):
+            for param_exp in param_list:
+                if isinstance(param_exp, ParameterExpression):
+                    template_params.update(param_exp.parameters)
+
+        circuit_params = set()
+        for param_list in (node.op.params for node in self.circuit_dag_dep.get_nodes()):
+            for param_exp in param_list:
+                if isinstance(param_exp, ParameterExpression):
+                    circuit_params.update(param_exp.parameters)
+
+        return len(template_params) > len(circuit_params)

releasenotes/notes/add-parameters-to-template-substitution-a1379cdbfcc10b5c.yaml

@@ -0,0 +1,74 @@
+---
+features:
+  - |
+    The :class:`.ParameterExpression` class is now allowed in the 
+    template optimization transpiler pass. An illustrative example 
+    of using :class:`.Parameter`\s is the following:
+
+    .. code-block:: 
+
+      from qiskit import QuantumCircuit, transpile, schedule
+      from qiskit.circuit import Parameter
+      
+      from qiskit.transpiler import PassManager
+      from qiskit.transpiler.passes import TemplateOptimization
+      
+      # New contributions to the template optimization
+      from qiskit.transpiler.passes.calibration import RZXCalibrationBuilder, rzx_templates
+      
+      from qiskit.test.mock import FakeCasablanca
+      backend = FakeCasablanca()
+      
+      phi = Parameter('φ')
+      
+      qc = QuantumCircuit(2)
+      qc.cx(0,1)
+      qc.p(2*phi, 1)
+      qc.cx(0,1)
+      print('Original circuit:')
+      print(qc)
+      
+      pass_ = TemplateOptimization(**rzx_templates.rzx_templates(['zz2']))
+      qc_cz = PassManager(pass_).run(qc)
+      print('ZX based circuit:')
+      print(qc_cz)
+      
+      # Add the calibrations
+      pass_ = RZXCalibrationBuilder(backend)
+      cal_qc = PassManager(pass_).run(qc_cz.bind_parameters({phi: 0.12}))
+      
+      # Transpile to the backend basis gates
+      cal_qct = transpile(cal_qc, backend)
+      qct = transpile(qc.bind_parameters({phi: 0.12}), backend)
+      
+      # Compare the schedule durations
+      print('Duration of schedule with the calibration:')
+      print(schedule(cal_qct, backend).duration)
+      print('Duration of standard with two CNOT gates:')
+      print(schedule(qct, backend).duration)
+          
+    outputs
+
+    .. parsed-literal::
+
+      Original circuit:
+      			 
+      q_0: ──■──────────────■──
+           ┌─┴─┐┌────────┐┌─┴─┐
+      q_1: ┤ X ├┤ P(2*φ) ├┤ X ├
+           └───┘└────────┘└───┘
+      ZX based circuit:
+      					 ┌─────────────┐            »
+      q_0: ────────────────────────────────────┤0            ├────────────»
+           ┌──────────┐┌──────────┐┌──────────┐│  Rzx(2.0*φ) │┌──────────┐»
+      q_1: ┤ Rz(-π/2) ├┤ Rx(-π/2) ├┤ Rz(-π/2) ├┤1            ├┤ Rx(-2*φ) ├»
+           └──────────┘└──────────┘└──────────┘└─────────────┘└──────────┘»
+      «                                                     
+      «q_0: ────────────────────────────────────────────────
+      «     ┌──────────┐┌──────────┐┌──────────┐┌──────────┐
+      «q_1: ┤ Rz(-π/2) ├┤ Rx(-π/2) ├┤ Rz(-π/2) ├┤ P(2.0*φ) ├
+      «     └──────────┘└──────────┘└──────────┘└──────────┘
+      Duration of schedule with the calibration:
+      1600
+      Duration of standard with two CNOT gates:
+      6848