Skip to content

Latest commit

 

History

History
320 lines (299 loc) · 18.3 KB

report.md

File metadata and controls

320 lines (299 loc) · 18.3 KB

Benchmark Report

Job Properties

Commits: d-netto/julia@c4a5dd937a3c135a134e91003e0ce2fd8e362fd9 vs JuliaLang/julia@51bb96857d26f67e62f0edc4fc4682a156cb3d08

Comparison Diff: link

Triggered By: link

Tag Predicate: !"scalar"

Results

Note: If Chrome is your browser, I strongly recommend installing the Wide GitHub extension, which makes the result table easier to read.

Below is a table of this job's results, obtained by running the benchmarks found in JuliaCI/BaseBenchmarks.jl. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.

The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value.

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["alloc", "arrays"] 1.06 (5%) ❌ 1.00 (1%)
["array", "cat", ("catnd", 5)] 1.07 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("collect", "StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}")] 1.77 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("collect", "Vector{Float64}")] 4.08 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_collect", "StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}")] 1.77 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_collect", "Vector{Float64}")] 2.32 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_indexing", "Vector{Float64}")] 1.38 (30%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_iteration", "StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}")] 1.23 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_iteration", "Vector{Float64}")] 1.42 (5%) ❌ 1.00 (1%)
["array", "equality", ("==", "BitArray")] 0.92 (5%) ✅ 1.00 (1%)
["array", "equality", ("==", "UnitRange{Int64}")] 0.91 (5%) ✅ 1.00 (1%)
["array", "equality", ("==", "Vector{Int64}")] 1.27 (5%) ❌ 1.00 (1%)
["array", "equality", ("isequal", "UnitRange{Int64}")] 0.82 (5%) ✅ 1.00 (1%)
["array", "equality", ("isequal", "Vector{Int64} isequal UnitRange{Int64}")] 0.88 (5%) ✅ 1.00 (1%)
["array", "equality", ("isequal", "Vector{Int64} isequal Vector{Float32}")] 0.94 (5%) ✅ 1.00 (1%)
["array", "equality", ("isequal", "Vector{Int64} isequal Vector{Int64}")] 1.07 (5%) ❌ 1.00 (1%)
["array", "growth", ("push_single!", 2048)] 0.94 (5%) ✅ 1.00 (1%)
["array", "growth", ("push_single!", 256)] 1.07 (5%) ❌ 1.00 (1%)
["array", "growth", ("push_single!", 8)] 1.07 (5%) ❌ 1.00 (1%)
["array", "setindex!", ("setindex!", 2)] 1.07 (5%) ❌ 1.00 (1%)
["array", "setindex!", ("setindex!", 4)] 0.93 (5%) ✅ 1.00 (1%)
["broadcast", "dotop", ("Float64", "(1000000,)", 2)] 1.14 (5%) ❌ 1.00 (1%)
["broadcast", "mix_scalar_tuple", (10, "scal_tup")] 1.12 (5%) ❌ 1.00 (1%)
["broadcast", "sparse", ("(1000, 1000)", 2)] 0.93 (5%) ✅ 1.00 (1%)
["broadcast", "typeargs", ("array", 3)] 1.06 (5%) ❌ 1.00 (1%)
["broadcast", "typeargs", ("tuple", 5)] 0.68 (5%) ✅ 1.00 (1%)
["collection", "deletion", ("Set", "Int", "filter!")] 1.75 (25%) ❌ 1.00 (1%)
["collection", "set operations", ("BitSet", "Int", "==", "self")] 0.63 (25%) ✅ 1.00 (1%)
["collection", "set operations", ("BitSet", "Int", "⊆", "Set")] 0.66 (25%) ✅ 1.00 (1%)
["collection", "set operations", ("BitSet", "Int", "⊆", "Vector")] 0.68 (25%) ✅ 1.00 (1%)
["dates", "parse", ("DateTime", "ISODateTimeFormat")] 1.06 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.5", "Vector{Float32}")] 1.07 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.5", "Vector{Float64}")] 1.05 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.5", "Vector{Int8}")] 1.10 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.5", "Vector{UInt8}")] 1.08 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.8", "Vector{Float32}")] 1.05 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.8", "Vector{Float64}")] 1.17 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.8", "Vector{Int64}")] 1.08 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.8", "Vector{Int8}")] 1.18 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.8", "Vector{UInt8}")] 1.08 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.95", "Vector{Float64}")] 1.14 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.95", "Vector{Int8}")] 1.10 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.95", "Vector{UInt8}")] 1.10 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.99", "Vector{Float32}")] 1.07 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.99", "Vector{UInt8}")] 1.09 (5%) ❌ 1.00 (1%)
["find", "findall", ("BitVector", "50-50")] 1.15 (5%) ❌ 1.00 (1%)
["find", "findall", ("Vector{Bool}", "10-90")] 1.05 (5%) ❌ 1.00 (1%)
["find", "findall", ("ispos", "Vector{Int64}")] 1.06 (5%) ❌ 1.00 (1%)
["find", "findall", ("ispos", "Vector{Int8}")] 1.06 (5%) ❌ 1.00 (1%)
["find", "findall", ("ispos", "Vector{UInt64}")] 1.06 (5%) ❌ 1.00 (1%)
["find", "findprev", ("ispos", "Vector{Float64}")] 1.13 (5%) ❌ 1.00 (1%)
["inference", "abstract interpretation", "broadcast"] 1.05 (5%) ❌ 1.00 (1%)
["inference", "allinference", "construct_ssa!"] 1.05 (5%) ❌ 1.00 (1%)
["inference", "optimization", "construct_ssa!"] 1.08 (5%) ❌ 1.00 (1%)
["inference", "optimization", "domsort_ssa!"] 1.10 (5%) ❌ 1.00 (1%)
["inference", "optimization", "quadratic"] 1.08 (5%) ❌ 1.00 (1%)
["io", "serialization", ("deserialize", "Matrix{Float64}")] 1.06 (5%) ❌ 1.00 (1%)
["io", "serialization", ("serialize", "Matrix{Float64}")] 1.13 (5%) ❌ 1.00 (1%)
["io", "skipchars"] 1.09 (5%) ❌ 1.00 (1%)
["linalg", "small exp #29116"] 0.95 (5%) ✅ 1.00 (1%)
["misc", "23042", "ComplexF32"] 1.08 (5%) ❌ 1.00 (1%)
["misc", "23042", "Float64"] 1.05 (5%) ❌ 1.00 (1%)
["misc", "iterators", "zip(1:1000)"] 0.94 (5%) ✅ 1.00 (1%)
["misc", "iterators", "zip(1:1000, 1:1000, 1:1000)"] 1.09 (5%) ❌ 1.00 (1%)
["misc", "repeat", (200, 1, 24)] 1.09 (5%) ❌ 1.00 (1%)
["problem", "simplex", "simplex"] 0.91 (5%) ✅ 1.00 (1%)
["random", "ranges", ("rand", "ImplicitRNG", "Int", "1:1000")] 0.75 (25%) ✅ 1.00 (1%)
["random", "types", ("rand", "ImplicitRNG", "Float64")] 0.68 (25%) ✅ 1.00 (1%)
["random", "types", ("rand", "ImplicitRNG", "ImplicitFloat64")] 0.68 (25%) ✅ 1.00 (1%)
["random", "types", ("rand", "ImplicitRNG", "Int64")] 0.74 (25%) ✅ 1.00 (1%)
["random", "types", ("randn", "ImplicitRNG", "Float64")] 0.72 (25%) ✅ 1.00 (1%)
["random", "types", ("randn", "ImplicitRNG", "ImplicitFloat64")] 0.73 (25%) ✅ 1.00 (1%)
["simd", ("Cartesian", "axpy!", "Int64", 2, 32)] 0.68 (20%) ✅ 1.00 (1%)
["simd", ("Cartesian", "conditional_loop!", "Int32", 3, 31)] 1.25 (20%) ❌ 1.00 (1%)
["simd", ("Cartesian", "manual_example!", "Float64", 3, 64)] 0.69 (20%) ✅ 1.00 (1%)
["simd", ("Cartesian", "manual_example!", "Int32", 2, 31)] 1.27 (20%) ❌ 1.00 (1%)
["simd", ("CartesianPartition", "conditional_loop!", "Int32", 3, 63)] 1.46 (20%) ❌ 1.00 (1%)
["simd", ("CartesianPartition", "conditional_loop!", "Int32", 4, 31)] 0.67 (20%) ✅ 1.00 (1%)
["simd", ("CartesianPartition", "manual_example!", "Float64", 3, 64)] 0.76 (20%) ✅ 1.00 (1%)
["sparse", "constructors", ("Diagonal", 10)] 1.09 (5%) ❌ 1.00 (1%)
["sparse", "constructors", ("Diagonal", 1000)] 1.09 (5%) ❌ 1.00 (1%)
["sparse", "constructors", ("IV", 1000)] 0.80 (5%) ✅ 1.00 (1%)
["sparse", "constructors", ("SymTridiagonal", 100)] 1.09 (5%) ❌ 1.00 (1%)
["sparse", "constructors", ("Tridiagonal", 100)] 1.05 (5%) ❌ 1.00 (1%)
["sparse", "sparse solves", "square system (ldlt), matrix rhs"] 1.10 (5%) ❌ 1.00 (1%)
["sparse", "sparse solves", "square system (ldlt), vector rhs"] 0.93 (5%) ✅ 1.00 (1%)
["string", "repeat", "repeat str len 16"] 0.93 (5%) ✅ 1.00 (1%)
["tuple", "linear algebra", ("matmat", "(16, 16)", "(16, 16)")] 0.90 (5%) ✅ 1.00 (1%)
["tuple", "linear algebra", ("matmat", "(4, 4)", "(4, 4)")] 0.74 (5%) ✅ 1.00 (1%)
["tuple", "linear algebra", ("matmat", "(8, 8)", "(8, 8)")] 0.85 (5%) ✅ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(16, 16)", "(16,)")] 1.13 (5%) ❌ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(2, 2)", "(2,)")] 1.07 (5%) ❌ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(4, 4)", "(4,)")] 1.09 (5%) ❌ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(8, 8)", "(8,)")] 0.94 (5%) ✅ 1.00 (1%)
["tuple", "reduction", ("sum", "(2, 2)")] 0.83 (5%) ✅ 1.00 (1%)
["tuple", "reduction", ("sum", "(8,)")] 1.10 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "*", "BigInt", "(false, false)")] 0.91 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "*", "BigInt", "(false, true)")] 0.91 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "*", "BigInt", "(true, true)")] 0.90 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "identity", "BigFloat", 0)] 1.07 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "identity", "BigInt", 0)] 0.92 (5%) ✅ 1.00 (1%)
["union", "array", ("collect", "all", "BigFloat", 0)] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("collect", "all", "Bool", 0)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "*", "BigInt", "(false, false)")] 0.90 (5%) ✅ 1.00 (1%)
["union", "array", ("map", "*", "BigInt", "(false, true)")] 0.92 (5%) ✅ 1.00 (1%)
["union", "array", ("map", "*", "BigInt", "(true, true)")] 0.91 (5%) ✅ 1.00 (1%)
["union", "array", ("map", "abs", "BigFloat", 1)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "identity", "BigFloat", 0)] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "identity", "Bool", 0)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "BigInt", "(false, false)")] 0.91 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "BigInt", "(false, true)")] 0.91 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "BigInt", "(true, true)")] 0.90 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_simplecopy", "BigFloat", 0)] 1.08 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_simplecopy", "BigInt", 0)] 0.89 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum", "Float32", 0)] 0.93 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum4", "ComplexF64", 1)] 1.35 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Nothing, ComplexF64}", 0)] 1.11 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "sum", "Bool", 0)] 1.09 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["alloc"]
  • ["array", "accumulate"]
  • ["array", "any/all"]
  • ["array", "bool"]
  • ["array", "cat"]
  • ["array", "comprehension"]
  • ["array", "convert"]
  • ["array", "equality"]
  • ["array", "growth"]
  • ["array", "index"]
  • ["array", "reductions"]
  • ["array", "reverse"]
  • ["array", "setindex!"]
  • ["array", "subarray"]
  • ["broadcast"]
  • ["broadcast", "dotop"]
  • ["broadcast", "fusion"]
  • ["broadcast", "mix_scalar_tuple"]
  • ["broadcast", "sparse"]
  • ["broadcast", "typeargs"]
  • ["collection", "deletion"]
  • ["collection", "initialization"]
  • ["collection", "iteration"]
  • ["collection", "optimizations"]
  • ["collection", "queries & updates"]
  • ["collection", "set operations"]
  • ["dates", "accessor"]
  • ["dates", "arithmetic"]
  • ["dates", "construction"]
  • ["dates", "conversion"]
  • ["dates", "parse"]
  • ["dates", "query"]
  • ["dates", "string"]
  • ["find", "findall"]
  • ["find", "findnext"]
  • ["find", "findprev"]
  • ["frontend"]
  • ["inference", "abstract interpretation"]
  • ["inference", "allinference"]
  • ["inference", "optimization"]
  • ["io", "array_limit"]
  • ["io", "read"]
  • ["io", "serialization"]
  • ["io"]
  • ["linalg", "arithmetic"]
  • ["linalg", "blas"]
  • ["linalg", "factorization"]
  • ["linalg"]
  • ["micro"]
  • ["misc"]
  • ["misc", "23042"]
  • ["misc", "afoldl"]
  • ["misc", "allocation elision view"]
  • ["misc", "bitshift"]
  • ["misc", "foldl"]
  • ["misc", "issue 12165"]
  • ["misc", "iterators"]
  • ["misc", "julia"]
  • ["misc", "parse"]
  • ["misc", "repeat"]
  • ["misc", "splatting"]
  • ["problem", "chaosgame"]
  • ["problem", "fem"]
  • ["problem", "go"]
  • ["problem", "grigoriadis khachiyan"]
  • ["problem", "imdb"]
  • ["problem", "json"]
  • ["problem", "laplacian"]
  • ["problem", "monte carlo"]
  • ["problem", "raytrace"]
  • ["problem", "seismic"]
  • ["problem", "simplex"]
  • ["problem", "spellcheck"]
  • ["problem", "stockcorr"]
  • ["problem", "ziggurat"]
  • ["random", "collections"]
  • ["random", "randstring"]
  • ["random", "ranges"]
  • ["random", "sequences"]
  • ["random", "types"]
  • ["shootout"]
  • ["simd"]
  • ["sort", "insertionsort"]
  • ["sort", "issorted"]
  • ["sort", "mergesort"]
  • ["sort", "quicksort"]
  • ["sparse", "arithmetic"]
  • ["sparse", "constructors"]
  • ["sparse", "index"]
  • ["sparse", "matmul"]
  • ["sparse", "sparse matvec"]
  • ["sparse", "sparse solves"]
  • ["sparse", "transpose"]
  • ["string", "==(::AbstractString, ::AbstractString)"]
  • ["string", "==(::SubString, ::String)"]
  • ["string", "findfirst"]
  • ["string"]
  • ["string", "readuntil"]
  • ["string", "repeat"]
  • ["tuple", "index"]
  • ["tuple", "linear algebra"]
  • ["tuple", "misc"]
  • ["tuple", "reduction"]
  • ["union", "array"]

Version Info

Primary Build

Julia Version 1.9.0-DEV.941
Commit c4a5dd937a (2022-07-08 00:44 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 20.04.4 LTS
  uname: Linux 5.4.0-113-generic #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3084 MHz      74504 s        199 s      23324 s   38349918 s          0 s
       #2  3020 MHz    2343411 s         63 s     114888 s   35997069 s          0 s
       #3  3053 MHz      68003 s         92 s      17611 s   38351212 s          0 s
       #4  2979 MHz      51063 s        134 s      17089 s   38356884 s          0 s
       #5  2958 MHz      63076 s         83 s      17550 s   38235070 s          0 s
       #6  3154 MHz      55090 s         29 s      17221 s   38366480 s          0 s
       #7  3024 MHz      56432 s         72 s      17826 s   38378974 s          0 s
       #8  3087 MHz      64134 s         94 s      17350 s   38362807 s          0 s
  Memory: 31.32082748413086 GB (15499.765625 MB free)
  Uptime: 3.8485528e6 sec
  Load Avg:  1.0  1.0  1.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.5 (ORCJIT, haswell)
  Threads: 1 on 8 virtual cores

Comparison Build

Julia Version 1.9.0-DEV.926
Commit 51bb96857d (2022-07-08 00:05 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 20.04.4 LTS
  uname: Linux 5.4.0-113-generic #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3005 MHz      75172 s        218 s      23506 s   38453878 s          0 s
       #2  2894 MHz    2442089 s         63 s     119057 s   35999230 s          0 s
       #3  2894 MHz      68056 s         92 s      17622 s   38456132 s          0 s
       #4  2866 MHz      51217 s        134 s      17111 s   38461623 s          0 s
       #5  2348 MHz      63153 s         83 s      17565 s   38339852 s          0 s
       #6  2666 MHz      55231 s         29 s      17238 s   38471326 s          0 s
       #7  3127 MHz      56954 s         72 s      17848 s   38483431 s          0 s
       #8  2903 MHz      64989 s         94 s      17375 s   38466928 s          0 s
  Memory: 31.32082748413086 GB (15212.21484375 MB free)
  Uptime: 3.85905376e6 sec
  Load Avg:  1.01  1.02  1.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.5 (ORCJIT, haswell)
  Threads: 1 on 8 virtual cores