Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate circuit properties as part of benchmarking runs #225

Open
bachase opened this issue Feb 12, 2025 · 0 comments
Open

Validate circuit properties as part of benchmarking runs #225

bachase opened this issue Feb 12, 2025 · 0 comments
Labels
feature New feature or request

Comments

@bachase
Copy link
Collaborator

bachase commented Feb 12, 2025

Describe the solution you'd like
As part of benchmarking, our goal is to have reasonable comparable settings across different compilers. One assumption today is that all compilers are targeting the same set of gates. In #201 we learned this was not the case. Although we are addressing in #224 , we should explicitly check any assumptions about the generated circuits during the benchmarking run to avoid us mistakenly having an unfair or at least unintended comparison.

Rather than having to write checks using each compiler's native capabilities, we would ideally convert to a common format (qasm?), and perform the checks there.

Suggested initial checks include

  • target gateset
  • qubit topology/layout

Other Considerations
It will be important to decide what to do with a benchmark run if a check fails - does the whole suite of benchmarks also fail? Or let successful ones complete and mark the run for that specific compiler + circuit as failed?

@bachase bachase added the feature New feature or request label Feb 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant