[Bug]: Probability Assertion Error causes Tests Error Locally

[Fe-r-oz]

What happened?

A bug happened! It's not a bug per se but it seems that probability assertion error which causes one test to fail when testing locally. That ends up in one test failing as shown below.

  Expression: unitary_fidelity(probs["transmon"], subspace = subspace) > 0.99
  Evaluated: 0.9033195454368972 > 0.99

Output attached below.

Version

main branch

What does this bug affect?

• quantum system construction
• problem setup
• problem solution
• problem performance
• solution analysis
• plotting
• documentation
• tests
• other (please specify below)

Other information

Julia Version:1.10.2+0.x64.linux.gnu

     Testing Running tests...
Test Summary:         | Pass  Total   Time
QuantumCollocation.jl |    8      8  58.9s
Subspace termination.
Linearly dependent generators.
Subspace termination.
Subspace termination.
Subspace termination.
Subspace termination.
Subspace termination.
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in scalarize at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/arrays.jl
┌ Warning: `similarterm` is deprecated, use `maketerm` instead.
│ See https://github.com/JuliaSymbolics/TermInterface.jl for details.
│ The present call can be replaced by
│ `maketerm(typeof(x), SymbolicUtils.BasicSymbolic, [op, args...], symtype, metadata)`
│   caller = kwcall(::@NamedTuple{metadata::Base.ImmutableDict{DataType, Any}}, ::typeof(SymbolicUtils.similarterm), x::SymbolicUtils.BasicSymbolic{Real}, op::Function, args::Vector{Any}) at types.jl:646
└ @ SymbolicUtils ~/.julia/packages/SymbolicUtils/qyMYa/src/types.jl:646
┌ Warning: `similarterm` is deprecated, use `maketerm` instead.
│ See https://github.com/JuliaSymbolics/TermInterface.jl for details.
│ The present call can be replaced by
│ `maketerm(typeof(x), SymbolicUtils.BasicSymbolic, [op, args...], symtype, metadata)`
│   caller = kwcall(::@NamedTuple{metadata::Nothing}, ::typeof(SymbolicUtils.similarterm), x::SymbolicUtils.BasicSymbolic{Real}, op::Function, args::Vector{Any}) at types.jl:646
└ @ SymbolicUtils ~/.julia/packages/SymbolicUtils/qyMYa/src/types.jl:646
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in diff2term at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/utils.jl
WARNING: Symbolics.istree is deprecated, use TermInterface.iscall instead.
  likely near /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/test/problems_test.jl:32
in diff2term at /home/Transcendence/.julia/packages/Symbolics/Eas9m/src/utils.jl
┌ Warning: `similarterm` is deprecated, use `maketerm` instead.
│ See https://github.com/JuliaSymbolics/TermInterface.jl for details.
│ The present call can be replaced by
│ `maketerm(typeof(x), SymbolicUtils.BasicSymbolic, [op, args...], symtype, metadata)`
│   caller = polyize(x::SymbolicUtils.BasicSymbolic{Real}, pvar2sym::Bijections.Bijection{Any, Any}, sym2term::Dict{SymbolicUtils.BasicSymbolic, Any}, vtype::Type, pow::Function, Fs::Type, recurse::Bool) at polyform.jl:121
└ @ SymbolicUtils ~/.julia/packages/SymbolicUtils/qyMYa/src/polyform.jl:121
┌ Warning: `similarterm` is deprecated, use `maketerm` instead.
│ See https://github.com/JuliaSymbolics/TermInterface.jl for details.
│ The present call can be replaced by
│ `maketerm(typeof(x), SymbolicUtils.BasicSymbolic, [op, args...], symtype, metadata)`
│   caller = simterm at polyform.jl:251 [inlined]
└ @ Core ~/.julia/packages/SymbolicUtils/qyMYa/src/polyform.jl:251
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint

******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit https://github.com/coin-or/Ipopt
******************************************************************************

applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
Robust and Subspace Templates: Test Failed at /home/Transcendence/Desktop/New/quantumcollocation/QuantumCollocation.jl/src/problem_templates/unitary_robustness_problem.jl:127
  Expression: unitary_fidelity(probs["transmon"], subspace = subspace) > 0.99
   Evaluated: 0.9033195454368972 > 0.99

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] top-level scope
   @ ~/Desktop/New/quantumcollocation/QuantumCollocation.jl/src/problem_templates/unitary_robustness_problem.jl:508
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
Total number of variables............................:     1253
                     variables with only lower bounds:        0
                variables with lower and upper bounds:      251
                     variables with only upper bounds:        0
Total number of equality constraints.................:     1100
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        1
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0


Number of Iterations....: 100

                                   (scaled)                 (unscaled)
Objective...............:   6.0473647465087007e-04    6.0473647465087007e-04
Dual infeasibility......:   6.6870891988902981e-04    6.6870891988902981e-04
Constraint violation....:   4.0494705235044615e-04    4.0494705235044615e-04
Variable bound violation:   0.0000000000000000e+00    0.0000000000000000e+00
Complementarity.........:   8.0741448491115168e-05    8.0741448491115168e-05
Overall NLP error.......:   6.6870891988902981e-04    6.6870891988902981e-04


Number of objective function evaluations             = 285
Number of objective gradient evaluations             = 71
Number of equality constraint evaluations            = 285
Number of inequality constraint evaluations          = 285
Number of equality constraint Jacobian evaluations   = 102
Number of inequality constraint Jacobian evaluations = 102
Number of Lagrangian Hessian evaluations             = 0
Total seconds in IPOPT                               = 3.481

EXIT: Maximum Number of Iterations Exceeded.
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
Total number of variables............................:      753
                     variables with only lower bounds:        0
                variables with lower and upper bounds:      251
                     variables with only upper bounds:        0
Total number of equality constraints.................:      650
Total number of inequality constraints...............:        0
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0


Number of Iterations....: 100

                                   (scaled)                 (unscaled)
Objective...............:   4.0609522136549493e-03    4.0609522136549493e-03
Dual infeasibility......:   3.4403057934066027e+01    3.4403057934066027e+01
Constraint violation....:   2.8679926841301828e-05    2.8679926841301828e-05
Variable bound violation:   0.0000000000000000e+00    0.0000000000000000e+00
Complementarity.........:   1.8368800199845776e-04    1.8368800199845776e-04
Overall NLP error.......:   3.4403057934066027e+01    3.4403057934066027e+01


Number of objective function evaluations             = 745
Number of objective gradient evaluations             = 101
Number of equality constraint evaluations            = 745
Number of inequality constraint evaluations          = 0
Number of equality constraint Jacobian evaluations   = 101
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations             = 100
Total seconds in IPOPT                               = 4.217

EXIT: Maximum Number of Iterations Exceeded.
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
...trajectories...
...constraints...
applying constraint: initial value of Ũ1
applying constraint: initial value of a1
applying constraint: initial value of Ũ2
applying constraint: initial value of a2
applying constraint: final value of a1
applying constraint: final value of a2
applying constraint: bounds on a1
applying constraint: bounds on dda1
applying constraint: bounds on a2
applying constraint: bounds on dda2
...trajectories...
...constraints...
applying constraint: initial value of Ũa
applying constraint: initial value of aa
applying constraint: initial value of Ũb
applying constraint: initial value of ab
applying constraint: final value of aa
applying constraint: final value of ab
applying constraint: bounds on aa
applying constraint: bounds on ddaa
applying constraint: bounds on ab
applying constraint: bounds on ddab
...trajectories...
...constraints...
applying constraint: initial value of Ũ1
applying constraint: initial value of a1
applying constraint: initial value of Ũ2
applying constraint: initial value of a2
applying constraint: final value of a1
applying constraint: final value of a2
applying constraint: bounds on a1
applying constraint: bounds on dda1
applying constraint: bounds on a2
applying constraint: bounds on dda2
...trajectories...
...constraints...
applying constraint: initial value of Ũ1
applying constraint: initial value of a1
applying constraint: initial value of Ũ2
applying constraint: initial value of a2
applying constraint: initial value of Ũ3
applying constraint: initial value of a3
applying constraint: final value of a1
applying constraint: final value of a2
applying constraint: final value of a3
applying constraint: bounds on a1
applying constraint: bounds on dda1
applying constraint: bounds on a2
applying constraint: bounds on dda2
applying constraint: bounds on a3
applying constraint: bounds on dda3
...trajectories...
...constraints...
applying constraint: initial value of Ũ1
applying constraint: initial value of a1
applying constraint: initial value of Ũ2
applying constraint: initial value of a2
applying constraint: final value of a1
applying constraint: final value of a2
applying constraint: bounds on a1
applying constraint: bounds on dda1
applying constraint: bounds on a2
applying constraint: bounds on dda2
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
...trajectories...
...constraints...
applying constraint: initial value of Ũ1
applying constraint: initial value of a1
applying constraint: initial value of Ũ2
applying constraint: initial value of a2
applying constraint: final value of a1
applying constraint: final value of a2
applying constraint: bounds on a1
applying constraint: bounds on dda1
applying constraint: bounds on a2
applying constraint: bounds on dda2
applying constraint: initial value of Ũ1
applying constraint: initial value of a1
applying constraint: final value of a1
applying constraint: bounds on a1
applying constraint: bounds on dda1
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: initial value of Ũ
applying constraint: initial value of a
applying constraint: final value of a
applying constraint: bounds on a
applying constraint: bounds on dda
applying constraint: bounds on Δt
applying constraint: time step all equal constraint
Test Summary:                                           | Pass  Fail  Total     Time
Package                                                 |  134     1    135  3m18.8s
  test/quantum_utils_test.jl                            |              None     0.1s
  test/quantum_system_utils_test.jl                     |   15           15     4.6s
  test/problems_test.jl                                 |    1            1    35.9s
  src/problem_templates/unitary_minimum_time_problem.jl |    3            3    58.0s
  src/problem_templates/unitary_robustness_problem.jl   |    4     1      5    58.0s
    Robust and Subspace Templates                       |    4     1      5    58.0s
  test/embedded_operators_test.jl                       |   14           14     0.7s
  src/problem_templates/unitary_smooth_pulse_problem.jl |    1            1     5.7s
  test/rollouts_test.jl                                 |   18           18     7.5s
  src/problem_templates/unitary_direct_sum_problem.jl   |   13           13    13.0s
  test/losses_test.jl                                   |   21           21     0.5s
  test/direct_sums_test.jl                              |   44           44    14.8s
ERROR: LoadError: Some tests did not pass: 134 passed, 1 failed, 0 errored, 0 broken.

[andgoldschmidt]

Good catch. We should not have testing which depends on random seed, which is what we currently have here. We can adjust this to guarantee convergence in some short number of iterations, by setting a_guess kwarg of the problem template to fill(π / 2,T) / T / Δt. This should correspond to starting the problem close to an X rotation, so that all the control needs to do is smooth over the pulse.

[Fe-r-oz]

Testing due to recent commits, there is another error that occur locally: It has to do with embedded_operators_test.jl file

EXIT: Maximum Number of Iterations Exceeded.
ERROR: LoadError: KeyError: key :CZ not found
Stacktrace:
  [1] getindex(h::Dict{Symbol, Matrix{ComplexF64}}, key::Symbol)
    @ Base ./dict.jl:498
  [2] top-level scope
    @ ~/Desktop/New/quantumcollocation/4/QuantumCollocation.jl/test/embedded_operators_test.jl:90
  [3] eval
    @ ./boot.jl:385 [inlined]
  [4] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2076
  [5] include_string(m::Module, txt::String, fname::String)
    @ Base ./loading.jl:2086
  [6] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
  [7] invokelatest
    @ ./essentials.jl:889 [inlined]
  [8] #3
    @ ~/.julia/packages/TestItemRunner/uEMJE/src/TestItemRunner.jl:102 [inlined]
  [9] withpath(f::TestItemRunner.var"#3#4"{String, String, Module}, path::String)
    @ TestItemRunner ~/.julia/packages/TestItemRunner/uEMJE/src/vendored_code.jl:7
 [10] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet)
    @ TestItemRunner ~/.julia/packages/TestItemRunner/uEMJE/src/TestItemRunner.jl:101
 [11] run_tests(path::String; filter::Nothing, verbose::Bool)
    @ TestItemRunner ~/.julia/packages/TestItemRunner/uEMJE/src/TestItemRunner.jl:185
 [12] run_tests(path::String)
    @ TestItemRunner ~/.julia/packages/TestItemRunner/uEMJE/src/TestItemRunner.jl:106
 [13] top-level scope
    @ ~/Desktop/New/quantumcollocation/4/QuantumCollocation.jl/test/runtests.jl:24
 [14] include(fname::String)
    @ Base.MainInclude ./client.jl:489
 [15] top-level scope
    @ none:6
in expression starting at /home/Transcendence/Desktop/New/quantumcollocation/4/QuantumCollocation.jl/test/embedded_operators_test.jl:90
in expression starting at /home/Transcendence/Desktop/New/quantumcollocation/4/QuantumCollocation.jl/test/runtests.jl:24
ERROR: Package QuantumCollocation errored during testing

[Fe-r-oz]

The recent error was because not pulling the recent commits, it's solved.

Test Summary:                                           | Pass  Fail  Total     Time
Package                                                 |  218     2    220  3m20.4s
  test/direct_sums_test.jl                              |   44           44    55.6s
  test/quantum_utils_test.jl                            |   71           71     9.6s
  test/rollouts_test.jl                                 |   18           18     7.2s
  test/problems_test.jl                                 |    1            1     4.5s
  src/problem_templates/unitary_minimum_time_problem.jl |    3            3    49.3s
  test/quantum_system_utils_test.jl                     |   15           15     4.8s
  src/problem_templates/unitary_robustness_problem.jl   |    3     2      5    57.4s
    Robust and Subspace Templates                       |    3     2      5    57.4s
  src/problem_templates/unitary_smooth_pulse_problem.jl |    1            1     5.3s
  test/losses_test.jl                                   |   21           21     0.7s
  test/embedded_operators_test.jl                       |   28           28     1.2s
  src/problem_templates/unitary_direct_sum_problem.jl   |   13           13     4.8s
ERROR: LoadError: Some tests did not pass: 218 passed, 2 failed, 0 errored, 0 broken.