-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] broadcast arguments mixin #7718
Conversation
Adding more mixins is going to exacerbate the performance regressions that we saw with the original PR, and I'm not certain what the benefit is. I don't see why this would need to be a mixin - it can be a regular method on the existing This said, I've actually been thinking about making a much larger restructuring of At the moment, there's a bunch of undefined assumptions (not all of which are compatible, I believe), and I can only assume it's buggy anyway: for example, In somewhat summary: the only change I think you need for your goals is simply to add a |
Pull Request Test Coverage Report for Build 2028275236
💛 - Coveralls |
This was intended as a way to facilitate code reuse across different families of In wall time, each of the regressions reported were <5ms with the exception of Also, agree this does seem like a very good time to at least move |
Mostly my issue is that mixins shouldn't be the default way to re-use code - that makes sense in a compiled language where there's no run-time penalty for extending the lookup paths, but in Python everything is dynamic and there are more options for importing code into classes. There's more unnecessary complications with mixins (we need to make sure every mixin defines If we want to share functions between classes, it can be as simple as having a module @staticmethod
def numpy_like(qargs, cargs):
qargs = [(qarg,) if isinstance(qarg, Qubit) else tuple(qarg) for qarg in qargs]
cargs = [(carg,) if isinstance(carg, Clbit) else tuple(carg) for carg in cargs]
all_args = qargs + cargs
split = len(qargs)
n_returns = max(all_args, key=len)
broadcast_args = []
for arg in all_args:
if len(arg) == n_returns:
broadcast_args.append(arg)
elif len(arg) == 1:
broadcast_args.append(arg * n_returns)
else:
raise CircuitError("incorrect lengths in broadcast")
for out_args in zip(*broadcast_args):
yield out_args[:split], out_args[split:]
@staticmethod
def none(qargs, cargs):
qargs, cargs = tuple(qargs), tuple(cargs)
if not (all(isinstance(qarg, Qubit) for qarg in qargs) and all(isinstance(carg, Clbit) for carg in cargs)):
raise CircuitError("this instruction does not support broadcasting")
yield qargs, cargs
... and then in the individual classes it just goes like class Operation(ABC):
broadcast_arguments = broadcast.none
...
class Instruction(Operation):
broadcast_arguments = broadcast.none
class Gate(Instruction):
broadcast_arguments = broadcast.numpy_like This has the same code cost to the author as a mixin, but it uses the standard lookup rules. Mixins are a good pattern when there's a few parts that need to interact with each other, but if it's just about adding a single function, it feels like binding in a single value has all the advantages and none of the downsides. For a major downside, consider this: In [1]: class InstructionBroadcastMixin:
...: @staticmethod
...: def broadcast():
...: return "instruction-like"
...: class GateBroadcastMixin:
...: @staticmethod
...: def broadcast():
...: return "gate-like"
...:
...: class Instruction(InstructionBroadcastMixin):
...: pass
...: class Gate(Instruction, GateBroadcastMixin):
...: pass
...:
...: (Instruction.broadcast(), Gate.broadcast())
Out[1]: ('instruction-like', 'instruction-like') Mixins don't play nicely with overloading, because the primary lookup for |
@jakelishman , @kdk, thanks for your comments (and sorry for a long reply). I have implemented the approach suggested by @jakelishman and have been running performance regression using If anyone is interested, the results are attached below. I have run the 3 versions (main vs. this PR vs. approach based on functions) three times each on both passes above, and I don't really see a noticeable difference between these; what makes the comparison a bit problematic is that I get slightly different results even when running the same thing multiple times (either because it runs on the laptop, or because the runs are inherently non-deterministic, though I thought that this was recently fixed using Are there any more tests I can do before we can decide which approach to go with? As @kdk said, we want to move auxiliary functionality completely out of
Regarding Python caveats, I believe mixins are expected to appear first, so there should be no problem to do things in the right order. Actually, @jakelishman, I may have misinterpreted your suggestion:
I have added the line in |
You're right, mixins should go first. My point is that it's just one example of increased cognitive load for something most people don't think about. It requires extra load to think "these classes are mixins that need to go first, then this class is the actual hierarchical parent, then these classes are additional interfaces that I define and need to go last", and then it makes it quite hard to look at a class and see what its logical parent really is. It's possible, but I really can't see any advantages in this case over direct method binding, and "making things harder to think about" is an important disadvantage of itself. I am slightly surprised that there seems to be no speed impact, though I do wonder if that would hold for much larger inheritance trees, especially including resolution of methods that are defined low in the tree. It's one more example in the quiver of the "no premature optimisation" crowd, though. Yeah, exactly, the example I gave was pretty verbatim of what I meant - methods should be bound in the class scope, not the instance scope, no matter where they come from. The reason is that class A:
def my_method(self):
pass is (very nearly) identical to doing def _my_method(self):
pass
class A:
my_method = _my_method That's why class Gate(Instruction, Operation):
broadcast_arguments = broadcasters.numpy_like
def __init__(self, ...): ... is the same as if it had been written as class Gate(Instruction, Operation):
def broadcast_arguments(...):
# ... the same code as `broadcasters.numpy_like` ...
def __init__(self, ...): .... Binding methods within I appreciate that the confusion on that point suggests it may also not be familiar to everyone, but I'd argue that regular method binding is still simpler long term and unlikely for people making new classes to get it wrong, since they'll just copy the existing styles and it doesn't need to interact with other elements in the MRO. |
@jakelishman, thanks for the explanations! I agree that with your approach we don't need to worry about inadvertent performance slowdown. Though I do believe that it's easier to look at class definition, e.g., at |
I think that mixins provide a cleaner way to extend a class interface, using the well known concept of inheritance. Furthermore, beyond just the discussion of style preference, I find 2 annoying disadvantages with class method binding approach:
In addition, using quite unique names in the arg broadcasting mixins, chances for hitting name collisions during MRO are probably practically 0 |
For point 1: I'm not sure I understand how looking for these methods is any harder than looking for a normal method? For point 2: if PyCharm can't complete a method bound in this way, it might be a bug - Jedi-based completers (IPython, Jupyter, my editor plugin, etc) don't seem to have a problem with it, though I might be missing some of what you mean. This binding method is also how some For name collisions: I'm not 100% sure what potential issues you had in mind, but I completely agree there's no chance of any unwanted collisions here. I do fully accept that mixins are useful in many situations, but I think they're for additive code only, not for situations where we need to be able to override them later. I'm not sure that the design of Python fits with using mixins to define simple functions that may need to be overridden at several different levels, though it may well in other more statically dispatched OOP languages. For a real example in the current codebase: class InstructionBroadcast:
def broadcast_arguments(): ...
class GateBroadcast:
def broadcast_arguments(): ...
class Instruction(InstructionBroadcast):
pass
class Gate(GateBroadcast, Instruction):
pass So far so good - However, now the user wants (or we want) to define a gate In [2]: class UserGate(InstructionBroadcast, Gate):
...: pass
...:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 class UserGate(InstructionBroadcast, Gate):
2 pass
TypeError: Cannot create a consistent method resolution
order (MRO) for bases InstructionBroadcast, Gate The grandchild class can't use the mixin
but this is invalid - you can't have the same class in the MRO twice. As far as I'm aware, there's no way around this with a mixin structure. The only alternative (I can think of - I could be wrong) is to manually define the class UserGate(Gate):
def broadcast_arguments(self, qargs, cargs):
return InstructionBroadcast.broadcast_arguments(self, qargs, cargs) which is quite similar to the method I suggested of directly binding the functions into the class, except more verbose, it doesn't inherit the docstring, and the code reader has to check for two possible definition patterns rather than just one. The last point is particularly confusing when some classes will use "lack of mixin" to mean "inherit", and others will use "lack of mixin" to mean "I'm overriding this with a grandparent method". This is a situation where classes may very well want to use the same code that a grandparent does - Footnotes
|
In the language of OOP design patterns, I'd call my suggested method a simpler version of composition (Python lets us compose without creating inner objects, unlike traditional OOP), as opposed to the mixin structure being an example of inheritance. If it matters, my understanding is that modern OOP design has been moving in the direction of re-use by composition over inheritance for a while, but I'm getting a little out of my wheelhouse here - I come from a background of procedural and then functional programming, not object-oriented. For "the well known concept of inheritance": in my mind at least, my suggested method does this as well - it really is identical to the current way that In [1]: def _f(self):
...: pass
...:
...: class A:
...: f = _f
...:
...: def g(self):
...: pass
...:
In [2]: A.f
Out[2]: <function __main__._f(self)>
In [3]: A.g
Out[3]: <function __main__.A.g(self)>
In [4]: A().f
Out[4]: <bound method _f of <__main__.A object at 0x000002131F64F8B0>>
In [5]: A().g
Out[5]: <bound method A.g of <__main__.A object at 0x000002131F74A3D0>> |
Thanks Jake. Your example with |
Thanks Eli. Sorry I wasn't able to work out the issue a bit sooner in the conversation to save time - I hadn't originally spotted it (and I wasn't 100% against mixins for this usage before that point, just trying to find simpler ways). It'd be good to talk in person just in general, but unfortunately it'll have to wait a few days more - I'm away from work finishing my PhD thesis, and I'm just commenting on some GitHub issues to take a break from writing every so often. I'll be at the dev meetup in April, though. |
Eli and Jake, thanks! As Jake's example with In a private message, Eli has also suggested an alternative approach. We would (1) create a class
(2) inherit
and (3) change all the existing
This works, however the extra function call does seems to slightly slow down things. In this new reimplementation, I have almost completely adopted Jake's solution, yet did put all the relevant functions in a single class, i.e. the code is essentially this:
Eli, Jake, what do you think? I have a few additional questions:
P.S. the reason why we are discussing broadcasting so much is that (I am hoping) whichever solution we adopt for this extra functionality will be also the solution to all other extra functionalities, |
I don't have any technical issues with this new style you're suggesting. I personally don't see the reason for the containing classes - it feels quite like using a workaround only needed in pure-OO languages, because in Python static methods in classes are not functionally any different to regular methods in modules (if you call the module On I think in your third bullet point you might have misunderstood me a little - I think we're much more in agreement already. I totally agree that class Operation(abc.ABC):
@abc.abstractmethod
def broadcast_arguments(self, qargs, cargs):
raise NotImplementedError
@abc.abstractmethod
@property
def _directive(self):
raise NotImplementedError
@property
def name(self):
raise NotImplementedError
# ... and so on ... This shouldn't have any impact on performance beyond what I agree with you that class Instruction(Operation):
_directive = False
broadcast_arguments = [bB]roadcast.no_broadcasting That satisfies the definition of the protocol and all the abstract methods, and shouldn't cause any errors or slowdown on instantiation. I think with broadcasting we need to be careful not to break API compatibility, so if we rename the method, perhaps I don't know the original reason it's called broadcasting, but I imagine it was by analogy to Numpy's terminology: |
Thanks Sasha. If we're going with method binding approach, I also don't see extra benefit with encapsulating static methods under a single class, rather than just putting those method in a single file. It may create a false perception that there is some intent to have those method share something at the class level ( Another point regarding Finally, a Mixin could be nice for argument broadcasting as a way to attach this extra functionality to circuit elements, but we agreed recently this has limited flexibility when it comes to choosing which broadcasting alg to use for which class. The approach I suggested as a replacement (single class with multiple instance methods, one for each broadcaster), is inspired by the Visitor design pattern. But this apparently also has performance penalty. My main point here though is that I'm curious to see how much run-time penalty this approach has vs using bound methods, I mean is it negligible or dramatic? I would argue that if we really want to have our code design process guided by performance considerations than we should at least measure performance in a somewhat more realistic setting, i.e with full-fledged circuits containing a lot of gates, rather than relying on microbenchmarks that may give as a skewed picture. |
I think BTW that we would benefit from planning a bit how |
Based on this discussion, I have added the proposed changes to the more general PR #7966. Closing this PR. |
Summary
Creating a mixin for the "broadcast arguments" functionality
Details and comments
This is the first (small) step towards adding a
Clifford
(and other objects) to aQuantumCircuit
natively, i.e. as anOperation
, without having to convert this clifford toInstruction
first (by synthesizing a quantum circuit for it).The "arguments broadcasting" functionality is a handy shortcut to add multiple gates at once, for instance
qc.cx(0, [1, 2, 3])
is a shortcut forCurrently
broadcast_arguments
is explicitly implemented inInstruction
,Barrier
,Delay
,Gate
,Measure
,Reset
andInitializer
. Some of these functions are very different, and some are exactly the same (e.g., for barrier and delay).I would like to get your feedback if this is the right approach to create the broadcaster mixin. I have moved all
broadcast_arguments
functionality to a separate file, and each relevant class (e.g.,Gate
) specifies the correctArgumentsBroadcaster
mixin in its definition.(Please ignore some printing statements and the additional test file for now).
One additional question is about naming things. Would it make sense to rename ArgumentsBroadcaster to something more intuitive, like ArgsExpander? Would it make sense to combine classes with the exact same functionality, like ArgumentsBroadcasterBarrier and ArgumentsBroadcasterDelay (and call it something suited for both)? Does it make sense to call ArgumentsBroadcasterGeneric for the functionality previously present in
Instruction
?