Make world-age increments explicit #56509
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The SparseArrays test failure is known - |
|
@nanosoldier |
|
I’m not opposed to this PR, but what’s the specific reason for introducing
(plus an increment after a That said, I think introducing |
aviatesk
reviewed
Nov 9, 2024
test/reflection.jl
Outdated
| @@ -777,6 +777,7 @@ faz1(x) = foo1(x) | |||
| @test faz1(1.0) == 1 | |||
| m = first(methods(bar1, Tuple{Int})) | |||
| Base.delete_method(m) | |||
There was a problem hiding this comment.
Does this mean that the result of Base.delete_method (and Base.delete_binding) won’t take effect until the user explicitly calls Core.@worldinc? That sounds like it would be a pretty breaking change. I agree with the idea of restricting the effect of :worldinc to the top level (for helping inference), but it seems like Base.delete_method might need some adjustments, such as calling @eval internally or something similar.
There was a problem hiding this comment.
Does this mean that the result of Base.delete_method (and Base.delete_binding) won’t take effect until the user explicitly calls Core.@WorldInc?
It takes effect immediately (in a new world age), but the effect is not visible in the current task until it raises it world age.
Calling @eval internally will not raise the world age, because it's not at top level. delete_method is mostly used internally by Revise which can take an appropriate adjustment. There could also be a delete_method macro that does the world age raising implicitly.
Two reasons:
|
|
@vtjnash suggests implicit world-age increments between the args of |
Forgot to mention that @vtjnash pointed out that I already made world age increment for macro purposes in #53515, so it would be inconsistent for it not to increment the world age for execution, which I found convincing. |
d085375 to
4330bad
Compare
|
The package evaluation job you requested has completed - possible new issues were detected. |
|
@nanosoldier |
|
The package evaluation job you requested has completed - possible new issues were detected. |
|
Alright, a fair number of tests with dependencies on JuliaInterpreter or Revise, which need to be taught about
|
Keno
added a commit
that referenced
this pull request
Nov 11, 2024
Split out from #56509 to facilitate adjusting downstream packages.
Keno
added a commit
that referenced
this pull request
Nov 11, 2024
This was part of #56509, but is an independent bugfix. The basic issue is that these macro were using `do` block internally. This is undesirable for test macros, because we would like them not to affect the behavior of what they're testing. E.g. right now: ``` julia> using Test julia> const x = 1 1 julia> @test_nowarn const x = 1 ERROR: syntax: `global const` declaration not allowed inside function around /home/keno/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:927 Stacktrace: [1] top-level scope @ REPL[3]:1 ``` This PR just writes out the try/finally manually, so the above works fine after this PR.
aviatesk
pushed a commit
that referenced
this pull request
Nov 11, 2024
This was part of #56509, but is an independent bugfix. The basic issue is that these macro were using `do` block internally. This is undesirable for test macros, because we would like them not to affect the behavior of what they're testing. E.g. right now: ``` julia> using Test julia> const x = 1 1 julia> @test_nowarn const x = 1 ERROR: syntax: `global const` declaration not allowed inside function around /home/keno/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:927 Stacktrace: [1] top-level scope @ REPL[3]:1 ``` This PR just writes out the try/finally manually, so the above works fine after this PR.
Keno
added a commit
that referenced
this pull request
Nov 12, 2024
Split out from #56509 to facilitate adjusting downstream packages.
vtjnash
reviewed
Nov 13, 2024
stdlib/REPL/src/REPL.jl
Outdated
| @@ -367,6 +367,7 @@ function eval_user_input(@nospecialize(ast), backend::REPLBackend, mod::Module) | |||
| for xf in backend.ast_transforms | |||
| ast = Base.invokelatest(xf, ast) | |||
| end | |||
| toplevel_eval_with_hooks(mod, Expr(:worldinc)) | |||
There was a problem hiding this comment.
This is a no-op (if the implementation is not broken)
Suggested change
| toplevel_eval_with_hooks(mod, Expr(:worldinc)) |
vtjnash
reviewed
Nov 13, 2024
There was a problem hiding this comment.
Overall SGTM (though needs a rebase against the parts that are merged already).
Looks like a lot of the needed added Core.@worldinc also will be already solved by making each toplevel / module statement continue to imply an updated world (since they already imply an updated macro world, and the statements are supposed to run in the world after that macro expansion, which wouldn't be possible to express with these explicit markers anyways)
One other remaining todo item is adding this to the Compiler verifier code. I don't know if we run any debug tests on PRs, but I believe we run those on nightly, and it likely should error on the unexpected Expr head.
Keno
added a commit
to JuliaDebug/JuliaInterpreter.jl
that referenced
this pull request
Nov 14, 2024
These were added in JuliaLang/julia#56523. This simply adds tracking for the world age to the interpreter, but does not use it for anything. JuliaInterpreter's current model of world age does not match Base anyway. We should fix that eventually, but it's probably easier to do that once JuliaLang/julia#56509 and binding partitions are merged.
4330bad to
3bbfc51
Compare
Keno
added a commit
to JuliaDebug/JuliaInterpreter.jl
that referenced
this pull request
Nov 14, 2024
* Handle `:latestworld` expr These were added in JuliaLang/julia#56523. This simply adds tracking for the world age to the interpreter, but does not use it for anything. JuliaInterpreter's current model of world age does not match Base anyway. We should fix that eventually, but it's probably easier to do that once JuliaLang/julia#56509 and binding partitions are merged. * 1.6 compat
Keno
added a commit
to timholy/Revise.jl
that referenced
this pull request
Nov 15, 2024
Upcoming base PR JuliaLang/julia#56509 makes precise when exactly world age increments automatically at top-level. Packages are mostly not supposed to notice, but of course Revise is a bit patholgical in that it does basically expect asynchronous changes to have top level effects. After that PR, that requires an explicit opt-in. This PR makes the appropriate adjustments to the tests.
Keno
added a commit
to timholy/Revise.jl
that referenced
this pull request
Nov 15, 2024
Upcoming base PR JuliaLang/julia#56509 makes precise when exactly world age increments automatically at top-level. Packages are mostly not supposed to notice, but of course Revise is a bit patholgical in that it does basically expect asynchronous changes to have top level effects. After that PR, that requires an explicit opt-in. This PR makes the appropriate adjustments to the tests.
|
@nanosoldier |
Keno
added a commit
to Keno/SparseArrays.jl
that referenced
this pull request
Nov 18, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
vtjnash
reviewed
Nov 20, 2024
d47b8ec to
c693de3
Compare
fingolfin
pushed a commit
to oscar-system/GenericCharacterTables.jl
that referenced
this pull request
Nov 28, 2024
This test currently relies on implicit world age increments at top level. We're re-evaluating where these go because julia is currently inconsistent about it in the interpreter, compiler and inference. To make sure this test keeps working on 1.12, add an explicit world age increment. See JuliaLang/julia#56509.
vtjnash
added a commit
that referenced
this pull request
Dec 4, 2024
- simplify float.jl loop not to call functions just defined to get back the value just stored there - add method to the correct checkbounds function (instead of a local) - missing world push/pop around jl_interpret_toplevel_thunk after #56509 - jl_lineno use maybe missed in #53799 - add some debugging dump for scm_to_julia_ mistakes
vtjnash
added a commit
that referenced
this pull request
Dec 4, 2024
- simplify float.jl loop not to call functions just defined to get back the value just stored there - add method to the correct checkbounds function (instead of a local) - missing world push/pop around jl_interpret_toplevel_thunk after #56509 - jl_lineno use maybe missed in #53799 - add some debugging dump for scm_to_julia_ mistakes
udesou
added a commit
to mmtk/julia
that referenced
this pull request
Dec 6, 2024
* Implement faster `issubset` for `CartesianIndices{N}` (#56282)
Co-authored-by: xili <xili@phas.ubc.ca>
* Improve doc example: Extracting the type parameter from a super-type (#55983)
Documentation describes the correct way of extracting the element type
of a supertype:
https://docs.julialang.org/en/v1/manual/methods/#Extracting-the-type-parameter-from-a-super-type
However, one of the examples to showcase this is nonsensical since it is
a union of multiple element types.
I have replaced this example with a union over the dimension.
Now, the `eltype_wrong` function still gives a similar error, yet the
correct way returns the unambiguous answer.
---------
Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com>
* llvmpasses: force vector width for compatibility with non-x86 hosts. (#56300)
The pipeline-prints test currently fails when running on an
aarch64-macos device:
```
/Users/tim/Julia/src/julia/test/llvmpasses/pipeline-prints.ll:309:23: error: AFTERVECTORIZATION: expected string not found in input
; AFTERVECTORIZATION: vector.body
^
<stdin>:2:40: note: scanning from here
; *** IR Dump Before AfterVectorizationMarkerPass on julia_f_199 ***
^
<stdin>:47:27: note: possible intended match here
; *** IR Dump Before AfterVectorizationMarkerPass on jfptr_f_200 ***
^
Input file: <stdin>
Check file: /Users/tim/Julia/src/julia/test/llvmpasses/pipeline-prints.ll
-dump-input=help explains the following input dump.
Input was:
<<<<<<
1: opt: WARNING: failed to create target machine for 'x86_64-unknown-linux-gnu': unable to get target for 'x86_64-unknown-linux-gnu', see --version and --triple.
2: ; *** IR Dump Before AfterVectorizationMarkerPass on julia_f_199 ***
check:309'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
3: define i64 @julia_f_199(ptr addrspace(10) noundef nonnull align 16 dereferenceable(40) %0) #0 !dbg !4 {
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4: top:
check:309'0 ~~~~~
5: %1 = call ptr @julia.get_pgcstack()
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6: %ptls_field = getelementptr inbounds ptr, ptr %1, i64 2
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7: %ptls_load45 = load ptr, ptr %ptls_field, align 8, !tbaa !8
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.
.
.
42:
check:309'0 ~
43: L41: ; preds = %L41.loopexit, %L17, %top
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
44: %value_phi10 = phi i64 [ 0, %top ], [ %7, %L17 ], [ %.lcssa, %L41.loopexit ]
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
45: ret i64 %value_phi10, !dbg !52
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
46: }
check:309'0 ~~
47: ; *** IR Dump Before AfterVectorizationMarkerPass on jfptr_f_200 ***
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:309'1 ? possible intended match
48: ; Function Attrs: noinline optnone
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
49: define nonnull ptr addrspace(10) @jfptr_f_200(ptr addrspace(10) %0, ptr noalias nocapture noundef readonly %1, i32 %2) #1 {
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
50: top:
check:309'0 ~~~~~
51: %3 = call ptr @julia.get_pgcstack()
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
52: %4 = getelementptr inbounds ptr addrspace(10), ptr %1, i32 0
check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.
.
.
>>>>>>
--
********************
Failed Tests (1):
Julia :: pipeline-prints.ll
```
The problem is that these tests assume x86_64, which fails because the
target isn't available, so it presumably uses the native target which
has different vectorization characteristics:
```
❯ ./usr/tools/opt --load-pass-plugin=libjulia-codegen.dylib -passes='julia' --print-before=AfterVectorization -o /dev/null ../../test/llvmpasses/pipeline-prints.ll
./usr/tools/opt: WARNING: failed to create target machine for 'x86_64-unknown-linux-gnu': unable to get target for 'x86_64-unknown-linux-gnu', see --version and --triple.
```
There's other tests that assume this (e.g. the `fma` cpufeatures one),
but they don't fail, so I've left them as is.
* Reduce generic matrix*vector latency (#56289)
```julia
julia> using LinearAlgebra
julia> A = rand(Int,4,4); x = rand(Int,4); y = similar(x);
julia> @time mul!(y, A, x, 2, 2);
0.330489 seconds (792.22 k allocations: 41.519 MiB, 8.75% gc time, 99.99% compilation time) # master
0.134212 seconds (339.89 k allocations: 17.103 MiB, 15.23% gc time, 99.98% compilation time) # This PR
```
Main changes:
- `generic_matvecmul!` and `_generic_matvecmul!` now accept `alpha` and
`beta` arguments instead of `MulAddMul(alpha, beta)`. The methods that
accept a `MulAddMul(alpha, beta)` are also retained for backward
compatibility, but these now forward `alpha` and `beta`, instead of the
other way around.
- Narrow the scope of the `@stable_muladdmul` applications. We now
construct the `MulAddMul(alpha, beta)` object only where it is needed in
a function call, and we annotate the call site with `@stable_muladdmul`.
This leads to smaller branches.
- Create a new internal function with methods for the `'N'`, `'T'` and
`'C'` cases, so that firstly, there's less code duplication, and
secondly, the `_generic_matvecmul!` method is now simple enough to
enable constant propagation. This eliminates the unnecessary branches,
and only the one that is taken is compiled.
Together, this reduces the TTFX substantially.
* Type `Base.is_interactive` as `Bool` (#56303)
Before, typing `Base.is_interactive = 7` would cause weird internal REPL
failures down the line. Now, it throws an InexactError and has no
impact.
* REPL: don't complete str and cmd macros when the input matches the internal name like `r_` to `r"` (#56254)
* fix REPL test if a "juliadev" directory exists in home (#56218)
* Fix trampoline warning on x86 as well (#56280)
* typeintersect: more fastpath to skip intersect under circular env (#56304)
fix #56040
* Preserve type in `first` for `OneTo` (#56263)
With this PR,
```julia
julia> first(Base.OneTo(10), 4)
Base.OneTo(4)
```
Previously, this would have used indexing to return a `UnitRange`. This
is probably the only way to slice a `Base.OneTo` and obtain a
`Base.OneTo` back.
* Matmul: dispatch on specific blas paths using an enum (#55002)
This expands on the approach taken by
https://github.com/JuliaLang/julia/pull/54552.
We pass on more type information to `generic_matmatmul_wrapper!`, which
lets us convert the branches to method dispatches. This helps spread the
latency around, so that instead of compiling all the branches in the
first call, we now compile the branches only when they are actually
taken. While this reduces the latency in individual branches, there is
no reduction in latency if all the branches are reachable.
```julia
julia> A = rand(2,2);
julia> @time A * A;
0.479805 seconds (809.66 k allocations: 40.764 MiB, 99.93% compilation time) # 1.12.0-DEV.806
0.346739 seconds (633.17 k allocations: 31.320 MiB, 99.90% compilation time) # This PR
julia> @time A * A';
0.030413 seconds (101.98 k allocations: 5.359 MiB, 98.54% compilation time) # v1.12.0-DEV.806
0.148118 seconds (219.51 k allocations: 11.652 MiB, 99.72% compilation time) # This PR
```
The latency is spread between the two calls here.
In fresh sessions:
```julia
julia> A = rand(2,2);
julia> @time A * A';
0.473630 seconds (825.65 k allocations: 41.554 MiB, 99.91% compilation time) # v1.12.0-DEV.806
0.490305 seconds (774.87 k allocations: 38.824 MiB, 99.90% compilation time) # This PR
```
In this case, both the `syrk` and `gemm` branches are reachable, so
there is no reduction in latency.
Analogously, there is a reduction in latency in the second set of matrix
multiplications where we call `symm!/hemm!` or `_generic_matmatmul`:
```julia
julia> using LinearAlgebra
julia> A = rand(2,2);
julia> @time Symmetric(A) * A;
0.711178 seconds (2.06 M allocations: 103.878 MiB, 2.20% gc time, 99.98% compilation time) # v1.12.0-DEV.806
0.540669 seconds (904.12 k allocations: 43.576 MiB, 2.60% gc time, 97.36% compilation time) # This PR
```
* Scaling `mul!` for generic `AbstractArray`s (#56313)
This improves performance in the scaling `mul!` for `StridedArray`s by
using loops instead of broadcasting.
```julia
julia> using LinearAlgebra
julia> A = zeros(200,200); C = similar(A);
julia> @btime mul!($C, $A, 1, 2, 2);
19.180 μs (0 allocations: 0 bytes) # nightly v"1.12.0-DEV.1479"
11.361 μs (0 allocations: 0 bytes) # This PR
```
The latency is reduced as well for the same reason.
```julia
julia> using LinearAlgebra
julia> A = zeros(2,2); C = similar(A);
julia> @time mul!(C, A, 1, 2, 2);
0.203034 seconds (522.94 k allocations: 27.011 MiB, 14.95% gc time, 99.97% compilation time) # nightly
0.034713 seconds (59.16 k allocations: 2.962 MiB, 99.91% compilation time) # This PR
```
Thirdly, I've replaced the `.*ₛ` calls by explicit branches. This fixes
the following:
```julia
julia> A = [zeros(2), zeros(2)]; C = similar(A);
julia> mul!(C, A, 1)
ERROR: MethodError: no method matching +(::Vector{Float64}, ::Bool)
```
After this,
```julia
julia> mul!(C, A, 1)
2-element Vector{Vector{Float64}}:
[0.0, 0.0]
[0.0, 0.0]
```
Also, I've added `@stable_muladdmul` annotations to the `generic_mul!`
call, but moved it within the loop to narrow its scope. This doesn't
increase the latency, while making the call type-stable.
```julia
julia> D = Diagonal(1:2); C = similar(D);
julia> @time mul!(C, D, 1, 2, 2);
0.248385 seconds (898.18 k allocations: 47.027 MiB, 12.30% gc time, 99.96% compilation time) # nightly
0.249940 seconds (919.80 k allocations: 49.128 MiB, 11.36% gc time, 99.99% compilation time) # This PR
```
* InteractiveUtils.jl: fixes issue where subtypes resolves bindings and causes deprecation warnings (#56306)
The current version of `subtypes` will throw deprecation errors even if
no one is using the deprecated bindings.
A similar bug was fixed in Aqua.jl -
https://github.com/JuliaTesting/Aqua.jl/pull/89/files
See discussion here:
- https://github.com/JuliaIO/ImageMagick.jl/issues/235 (for identifying
the problem)
- https://github.com/simonster/Reexport.jl/issues/42 (for pointing to
the issue in Aqua.jl)
- https://github.com/JuliaTesting/Aqua.jl/pull/89/files (for the fix in
Aqua.jl)
This adds the `isbindingresolved` test to the `subtypes` function to
avoid throwing deprecation warnings. It also adds a test to check that
this doesn't happen.
---
On the current master branch (before the fix), the added test shows:
```
WARNING: using deprecated binding InternalModule.MyOldType in OuterModule.
, use MyType instead.
Subtypes and deprecations: Test Failed at /home/dgleich/devextern/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:932
Expression: isempty(stderr_content)
Evaluated: isempty("WARNING: using deprecated binding InternalModule.MyOldType in OuterModule.\n, use MyType instead.\n")
Test Summary: | Fail Total Time
Subtypes and deprecations | 1 1 2.8s
ERROR: LoadError: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.
in expression starting at /home/dgleich/devextern/julia/stdlib/InteractiveUtils/test/runtests.jl:841
ERROR: Package InteractiveUtils errored during testing
```
---
Using the results of this pull request:
```
@test_nowarn subtypes(Integer);
```
passes without error. The other tests pass too.
* [CRC32c] Support AbstractVector{UInt8} as input (#56164)
This is a similar PR to https://github.com/JuliaIO/CRC32.jl/pull/12
I added a generic fallback method for `AbstractVector{UInt8}` similar to
the existing generic `IO` method.
Co-authored-by: Steven G. Johnson <stevenj@mit.edu>
* Put `jl_gc_new_weakref` in a header file again (#56319)
* use textwidth for string display truncation (#55442)
It makes a big difference when displaying strings that have width-2 or
width-0 characters.
* Use `pwd()` as the default directory to walk in `walkdir` (#55550)
* Reset mtime of BOLTed files to prevent make rebuilding targets (#55587)
This simplifies the `finish_stage` rule.
Co-authored-by: Zentrik <Zentrik@users.noreply.github.com>
* add docstring note about `displaysize` and `IOContext` with `context` (#55510)
* LinearAlgebra: replace some hardcoded loop ranges with axes (#56243)
These are safer in general, as well as easier to read.
Also, narrow the scopes of some `@inbounds` annotations.
* inference: fix `[modifyfield!|replacefield!]_tfunc`s (#56310)
Currently the following code snippet results in an internal error:
```julia
julia> func(x) = @atomic :monotonic x[].count += 1;
julia> let;Base.Experimental.@force_compile
x = Ref(nothing)
func(x)
end
Internal error: during type inference of
...
```
This issue is caused by the incorrect use of `_fieldtype_tfunc(𝕃, o, f)`
within `modifyfield!_tfunc`, specifically because `o` should be
`widenconst`ed, but it isn’t. By using `_fieldtype_tfunc` correctly, we
can avoid the error through error-catching in `abstract_modifyop!`. This
commit also includes a similar fix for `replacefield!_tfunc` as well.
* inference: don't allow `SSAValue`s in assignment lhs (#56314)
In `InferenceState` the lhs of a `:=` expression should only contain
`GlobalRef` or `SlotNumber` and no other IR elements. Currently when
`SSAValue` appears in `lhs`, the invalid assignment effect is somehow
ignored, but this is incorrect anyway, so this commit removes that
check. Since `SSAValue` should not appear in `lhs` in the first place,
this is not a significant change though.
* Fix `unsafe_read` for `IOBuffer` with non dense data (#55776)
Fixes one part of #54636
It was only safe to use the following if `from.data` was a dense vector
of bytes.
```julia
GC.@preserve from unsafe_copyto!(p, pointer(from.data, from.ptr), adv)
```
This PR adds a fallback suggested by @matthias314 in
https://discourse.julialang.org/t/copying-bytes-from-abstractvector-to-ptr/119408/7
* support `isless` for zero-dimensional `AbstractArray`s (#55772)
Fixes #55771
* inference: don't add backdge when `applicable` inferred to return `Bool` (#56316)
Also just as a minor backedge reduction optimization, this commit avoids
adding backedges when `applicable` is inferred to return `::Bool`.
* Mark `require_one_based_indexing` and `has_offset_axes` as public (#56196)
The discussion here mentions `require_one_based_indexing` being part of
the public API: https://github.com/JuliaLang/julia/pull/43263
Both functions are also documented (albeit in the dev docs):
* `require_one_based_indexing`:
https://docs.julialang.org/en/v1/devdocs/offset-arrays/#man-custom-indices
* `has_offset_axes`:
https://docs.julialang.org/en/v1/devdocs/offset-arrays/#For-objects-that-mimic-AbstractArray-but-are-not-subtypes
Towards https://github.com/JuliaLang/julia/issues/51335.
---------
Co-authored-by: Matt Bauman <mbauman@gmail.com>
* Avoid some allocations in various `println` methods (#56308)
* Add a developer documentation section to the `LinearAlgebra` docs (#56324)
Functions that are meant for package developers may go here, instead of
the main section that is primarily for users.
* drop require lock when not needed during loading to allow parallel precompile loading (#56291)
Fixes `_require_search_from_serialized` to first acquire all
start_loading locks (using a deadlock-free batch-locking algorithm)
before doing stalechecks and the rest, so that all the global
computations happen behind the require_lock, then the rest can happen
behind module-specific locks, then (as before) extensions can be loaded
in parallel eventually after `require` returns.
* Make `String(::Memory)` copy (#54457)
A more targeted fix of #54369 than #54372
Preserves the performance improvements added in #53962 by creating a new
internal `_unsafe_takestring!(v::Memory{UInt8})` function that does what
`String(::Memory{UInt8})` used to do.
* 🤖 [master] Bump the Pkg stdlib from 799dc2d54 to 116ba910c (#56336)
Stdlib: Pkg
URL: https://github.com/JuliaLang/Pkg.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 799dc2d54
New commit: 116ba910c
Julia version: 1.12.0-DEV
Pkg version: 1.12.0
Bump invoked by: @IanButterworth
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/Pkg.jl/compare/799dc2d54c4e809b9779de8c604564a5b3befaa0...116ba910c74ab565d348aa8a50d6dd10148f11ab
```
$ git log --oneline 799dc2d54..116ba910c
116ba910c fix Base.unreference_module call (#4057)
6ed1d2f40 do not show right hand progress without colors (#4047)
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* Wall-time/all tasks profiler (#55889)
One limitation of sampling CPU/thread profiles, as is currently done in
Julia, is that they primarily capture samples from CPU-intensive tasks.
If many tasks are performing IO or contending for concurrency primitives
like semaphores, these tasks won’t appear in the profile, as they aren't
scheduled on OS threads sampled by the profiler.
A wall-time profiler, like the one implemented in this PR, samples tasks
regardless of OS thread scheduling. This enables profiling of IO-heavy
tasks and detecting areas of heavy contention in the system.
Co-developed with @nickrobinson251.
* recommend explicit `using Foo: Foo, ...` in package code (was: "using considered harmful") (#42080)
I feel we are heading up against a "`using` crisis" where any new
feature that is implemented by exporting a new name (either in Base or a
package) becomes a breaking change. This is already happening
(https://github.com/JuliaGPU/CUDA.jl/pull/1097,
https://github.com/JuliaWeb/HTTP.jl/pull/745) and as projects get bigger
and more names are exported, the likelihood of this rapidly increases.
The flaw in `using Foo` is fundamental in that you cannot lexically see
where a name comes from so when two packages export the same name, you
are screwed. Any code that relies on `using Foo` and then using an
exported name from `Foo` is vulnerable to another dependency exporting
the same name.
Therefore, I think we should start to strongly discourage the use of
`using Foo` and only recommend `using Foo` for ephemeral work (e.g. REPL
work).
---------
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
Co-authored-by: Mason Protter <mason.protter@icloud.com>
Co-authored-by: Max Horn <max@quendi.de>
Co-authored-by: Matt Bauman <mbauman@juliahub.com>
Co-authored-by: Alex Arslan <ararslan@comcast.net>
Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
Co-authored-by: Neven Sajko <s@purelymail.com>
* Change some hardcoded loop ranges to axes in dense linalg functions (#56348)
These should be safer in general, and are also easier to reason about.
* Make `LinearAlgebra.haszero` public (#56223)
The trait `haszero` is used to check if a type `T` has a unique zero
defined using `zero(T)`. This lets us dispatch to optimized paths
without losing generality. This PR makes the function public so that
this may be extended by packages (such as `StaticArrays`).
* remove spurious parens in profiler docs (#56357)
* Fix `log_quasitriu` for internal scaling `s=0` (#56311)
This PR is a potential fix for #54833.
## Description
The function
https://github.com/JuliaLang/julia/blob/2a06376c18afd7ec875335070743dcebcd85dee7/stdlib/LinearAlgebra/src/triangular.jl#L2220
computes $\boldsymbol{A}^{\dfrac{1}{2^s}} - \boldsymbol{I}$ for a
real-valued $2\times 2$ matrix $\boldsymbol{A}$ using Algorithm 5.1 in
[R1]. However, the algorithm in [R1] as well as the above function do
not handle the case $s=0.$ This fix extends the function to compute
$\boldsymbol{A}^{\dfrac{1}{2^s}} - \boldsymbol{I} \Bigg|_{s=0} =
\boldsymbol{A} - \boldsymbol{I}.$
## Checklist
- [X] Fix code: `stdlib\LinearAlgebra\src\triangular.jl` in function
`_sqrt_pow_diag_block_2x2!(A, A0, s)`.
- [X] Add test case: `stdlib\LinearAlgebra\test\triangular.jl`.
- [X] Update `NEWS.md`.
- [X] Testing and self review.
| Tag | Reference |
| --- | --- |
| <nobr>[R1]</nobr> | Al-Mohy, Awad H. and Higham, Nicholas J. "Improved
Inverse Scaling and Squaring Algorithms for the Matrix Logarithm", 2011,
url: https://eprints.maths.manchester.ac.uk/1687/1/paper11.pdf |
---------
Co-authored-by: Daniel Karrasch <daniel.karrasch@posteo.de>
Co-authored-by: Oscar Smith <oscardssmith@gmail.com>
* loading: clean up more concurrency issues (#56329)
Guarantee that `__init__` runs before `using` returns. Could be slightly
breaking for people that do crazy things inside `__init__`, but just
don't do that. Since extensions then probably load after `__init__` (or
at least, run their `__init__` after), this is a partial step towards
changing things so that extensions are guaranteed to load if using all
of their triggers before the corresponding `using` returns
Fixes #55556
* make `_unsetindex` fast for isbits eltype (#56364)
fixes
https://github.com/JuliaLang/julia/issues/56359#issuecomment-2441537634
```
using Plots
function f(n)
a = Vector{Int}(undef, n)
s = time_ns()
resize!(a, 8)
time_ns() - s
end
x = 8:10:1000000
y = f.(x)
plot(x, y)
```
* improved `eltype` for `flatten` with tuple argument (#55946)
We have always had
```
julia> t = (Int16[1,2], Int32[3,4]); eltype(Iterators.flatten(t))
Any
```
With this PR, the result is `Signed` (`promote_typejoin` applied to the
element types of the tuple elements).
The same applies to `NamedTuple`:
```
julia> nt = (a = [1,2], b = (3,4)); eltype(Iterators.flatten(nt))
Any # old
Int64 # new
```
* Reland "Reroute (Upper/Lower)Triangular * Diagonal through __muldiag #55984" (#56270)
This relands #55984 which was reverted in #56267. Previously, in #55984,
the destination in multiplying triangular matrices with diagonals was
also assumed to be triangular, which is not necessarily the case in
`mul!`. Tests for this case, however, were being run
non-deterministically, so this wasn't caught by the CI runs.
This improves performance:
```julia
julia> U = UpperTriangular(rand(100,100)); D = Diagonal(rand(size(U,2))); C = similar(U);
julia> @btime mul!($C, $D, $U);
1.517 μs (0 allocations: 0 bytes) # nightly
1.116 μs (0 allocations: 0 bytes) # This PR
```
* Add one-arg `norm` method (#56330)
This reduces the latency of `norm` calls, as the single-argument method
lacks branches and doesn't use aggressive constant propagation, and is
therefore simpler to compile. Given that a lot of `norm` calls use
`p==2`, it makes sense for us to reduce the latency on this call.
```julia
julia> using LinearAlgebra
julia> A = rand(2,2);
julia> @time norm(A);
0.247515 seconds (390.09 k allocations: 19.993 MiB, 33.57% gc time, 99.99% compilation time) # master
0.067201 seconds (121.24 k allocations: 6.067 MiB, 99.98% compilation time) # this PR
```
An example of an improvement in ttfx because of this:
```julia
julia> A = rand(2,2);
julia> @time A ≈ A;
0.556475 seconds (1.16 M allocations: 59.949 MiB, 24.14% gc time, 100.00% compilation time) # master
0.333114 seconds (899.85 k allocations: 46.574 MiB, 8.11% gc time, 99.99% compilation time) # this PR
```
* fix a forgotten rename `readuntil` -> `copyuntil` (#56380)
Fixes https://github.com/JuliaLang/julia/issues/56352, with the repro in
that issue:
```
Master:
1.114874 seconds (13.01 M allocations: 539.592 MiB, 3.80% gc time)
After:
0.369492 seconds (12.99 M allocations: 485.031 MiB, 10.73% gc time)
1.10:
0.341114 seconds (8.36 M allocations: 454.242 MiB, 2.69% gc time)
```
* remove unnecessary operations from `typejoin_union_tuple` (#56379)
Removes the unnecessary call to `unwrap_unionall` and type assertion.
* precompile: fix performance issues with IO (#56370)
The string API here rapidly becomes unusably slow if dumping much debug
output during precompile. Fix the design here to use an intermediate IO
instead to prevent that.
* cache the `find_all_in_cache_path` call during parallel precompilation (#56369)
Before (in an environment with DifferentialEquations.jl):
```julia
julia> @time Pkg.precompile()
0.733576 seconds (3.44 M allocations: 283.676 MiB, 6.24% gc time)
julia> isfile_calls[1:10]
10-element Vector{Pair{String, Int64}}:
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/Printf/3FQLY_zHycD.ji" => 178
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/Printf/3FQLY_xxrt3.ji" => 178
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/Dates/p8See_xxrt3.ji" => 158
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/Dates/p8See_zHycD.ji" => 158
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/TOML/mjrwE_zHycD.ji" => 155
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/TOML/mjrwE_xxrt3.ji" => 155
"/home/kc/.julia/compiled/v1.12/Preferences/pWSk8_4Qv86.ji" => 152
"/home/kc/.julia/compiled/v1.12/Preferences/pWSk8_juhqb.ji" => 152
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/StyledStrings/UcVoM_zHycD.ji" => 144
"/home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/StyledStrings/UcVoM_xxrt3.ji" => 144
```
After:
```julia
julia> @time Pkg.precompile()
0.460077 seconds (877.59 k allocations: 108.075 MiB, 4.77% gc time)
julia> isfile_calls[1:10]
10-element Vector{Pair{String, Int64}}:
"/tmp/jl_a5xFWK/Project.toml" => 15
"/tmp/jl_a5xFWK/Manifest.toml" => 7
"/home/kc/.julia/registries/General.toml" => 6
"/home/kc/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/Markdown/src/Markdown.jl"
=> 3
"/home/kc/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/Serialization/src/Serialization.jl"
=> 3
"/home/kc/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/Distributed/src/Distributed.jl"
=> 3
"/home/kc/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/UUIDs/src/UUIDs.jl"
=> 3
"/home/kc/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LibCURL/src/LibCURL.jl"
=> 3
```
Performance is improved and we are not calling `isfile` on a bunch of the same ji files hundreds times.
Benchmark is made on a linux machine so performance diff should be a lot better on Windows where these `isfile_casesensitive` call is much more expensive.
Fixes https://github.com/JuliaLang/julia/issues/56366
---------
Co-authored-by: KristofferC <kristoffer.carlsson@juliacomputing.com>
Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
* [docs] Fix note admonition in llvm-passes.md (#56392)
At the moment this is rendered incorrectly:
https://docs.julialang.org/en/v1.11.1/devdocs/llvm-passes/#JuliaLICM
* structure-preserving broadcast for `SymTridiagonal` (#56001)
With this PR, certain broadcasting operations preserve the structure of
a `SymTridiagonal`:
```julia
julia> S = SymTridiagonal([1,2,3,4], [1,2,3])
4×4 SymTridiagonal{Int64, Vector{Int64}}:
1 1 ⋅ ⋅
1 2 2 ⋅
⋅ 2 3 3
⋅ ⋅ 3 4
julia> S .* 2
4×4 SymTridiagonal{Int64, Vector{Int64}}:
2 2 ⋅ ⋅
2 4 4 ⋅
⋅ 4 6 6
⋅ ⋅ 6 8
```
This was deliberately disabled on master, but I couldn't find any test
that fails if this is enabled.
* 🤖 [master] Bump the Pkg stdlib from 116ba910c to 9f8e11a4c (#56386)
Stdlib: Pkg
URL: https://github.com/JuliaLang/Pkg.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 116ba910c
New commit: 9f8e11a4c
Julia version: 1.12.0-DEV
Pkg version: 1.12.0
Bump invoked by: @IanButterworth
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/Pkg.jl/compare/116ba910c74ab565d348aa8a50d6dd10148f11ab...9f8e11a4c0efb3b68a1e25a33f372f398c89cd66
```
$ git log --oneline 116ba910c..9f8e11a4c
9f8e11a4c strip out tree_hash for stdlibs that have have been freed in newer julia versions (#4062)
c0df25a47 rm dead code (#4061)
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* load extensions with fewer triggers earlier (#49891)
Aimed to support the use case in
https://github.com/JuliaLang/julia/issues/48734#issuecomment-1554626135.
https://github.com/KristofferC/ExtSquared.jl is an example, see
specifically
https://github.com/KristofferC/ExtSquared.jl/blob/ded7c57d6f799674e3310b8174dfb07591bbe025/ext/BExt.jl#L4.
I think this makes sense, happy for a second pair of eyes though.
cc @termi-official
---------
Co-authored-by: KristofferC <kristoffer.carlsson@juliacomputing.com>
Co-authored-by: Cody Tapscott <84105208+topolarity@users.noreply.github.com>
* Dispatch in generic_matmatmul (#56384)
Replacing the branches by dispatch reduces latency, presumably because
there's less dead code in the method.
```julia
julia> using LinearAlgebra
julia> A = rand(Int,2,2); B = copy(A); C = similar(A);
julia> @time mul!(C, A, B, 1, 2);
0.363944 seconds (1.65 M allocations: 84.584 MiB, 37.57% gc time, 99.99% compilation time) # master
0.102676 seconds (176.55 k allocations: 8.904 MiB, 27.04% gc time, 99.97% compilation time) # this PR
```
The latency is now distributed between the different branches:
```julia
julia> @time mul!(C, A, B, 1, 2);
0.072441 seconds (176.55 k allocations: 8.903 MiB, 99.97% compilation time)
julia> @time mul!(C, A', B, 1, 2);
0.085817 seconds (116.44 k allocations: 5.913 MiB, 99.96% compilation time: 4% of which was recompilation)
julia> @time mul!(C, A', B', 1, 2);
0.345337 seconds (1.07 M allocations: 54.773 MiB, 25.77% gc time, 99.99% compilation time: 40% of which was recompilation)
```
It would be good to look into why there's recompilation in the last
case, but the branch is less commonly taken than the others that have
significantly lower latency after this PR.
* Add `atol` to addmul tests (#56210)
This avoids the issues as in
https://github.com/JuliaLang/julia/issues/55781 and
https://github.com/JuliaLang/julia/issues/55779 where we compare small
numbers using a relative tolerance. Also, in this PR, I have added an
extra test, so now we compare both `A * B * alpha + C * beta` and `A * B
* alpha - C * beta` with the corresponding in-place versions. The idea
is that if the terms `A * B * alpha` and ` C * beta` have similar
magnitudes, at least one of the two expressions will usually result in a
large enough number that may be compared using a relative tolerance.
I am unsure if the `atol` chosen here is optimal, as I have ballparked
it to use the maximum `eps` by looking at all the `eltype`s involved.
Fixes #55781
Fixes #55779
* Export jl_gc_new_weakref again via julia.h (#56373)
This is how it used for at least Julia 1.0 - 1.11
Closes #56367
* InteractiveUtils: define `InteractiveUtils.@code_ircode` (#56390)
* Fix some missing write barriers and add some helpful comments (#56396)
I was trying some performance optimization which didn't end up working
out, but in the process I found two missing write barriers and added
some helpful comments for future readers, so that part is probably still
useful.
* compiler: fix specialization mistake introduced by #40985 (#56404)
Hopefully there aren't any others like this hiding around? Not useful to
make a new closure for every method that we inline, since we just called
`===` inside it
* Avoid racy double-load of binding restriction in `import_module` (#56395)
Fixes #56333
* define `InteractiveUtils.@infer_[return|exception]_type` (#56398)
Also simplifies the definitions of `@code_typed` and the other similar
macros.
* irinterp: set `IR_FLAG_REFINED` for narrowed `PhiNode`s (#56391)
`adce_pass!` can transform a `Union`-type `PhiNode` into a narrower
`PhiNode`, but in such cases, the `IR_FLAG_REFINED` flag isn’t set on
that `PhiNode` statement. By setting this flag, irinterp can perform
statement reprocessing using the narrowed `PhiNode`, enabling type
stability in cases like JuliaLang/julia#56387.
- fixes JuliaLang/julia#56387
* document isopen(::Channel) (#56376)
This PR has two purposes --
1) Add some documentation for public API
2) Add a small note about a footgun I've hit a few times: `!isopen(ch)`
does not mean that you are "done" with the channel because buffered
channels can still have items left in them that need to be taken.
---------
Co-authored-by: CY Han <cyhan.dev@outlook.com>
* Make build system respect `FORCE_COLOR` and `NO_COLOR` settings (#56346)
Follow up to #53742, but for the build system. CC: @omus.
* Add `edges` vector to CodeInstance/CodeInfo to keep backedges as edges (#54894)
Appears to add about 11MB (128MB to 139MB) to the system image, and to
decrease the stdlib size by 55 MB (325MB to 270MB), so seems overall
favorable right now. The edges are computed following the encoding
<https://hackmd.io/sjPig55kS4a5XNWC6HmKSg?both#Edges-Encoding> to
correctly reflect the backedges.
Co-authored-by: Shuhei Kadowaki <aviatesk@gmail.com>
* docs: remove `dirname.c` from THIRDPARTY file (#56413)
- `dirname.c` was removed by
https://github.com/JuliaLang/julia/commit/c2cec7ad57102e4fbb733b8fb79d617a9524f0ae
* Allow ext → ext dependency if triggers are a strict superset (#56368) (#56402)
Forward port of #56368 - this was a pretty clean port, so it should be
good to go once tests pass.
* [docs] Fix rendering of warning admonition in llvm passes page (#56412)
Follow up to #56392: also the warning in
https://docs.julialang.org/en/v1.11.1/devdocs/llvm-passes/#Multiversioning
is rendered incorrectly because of a missing space.
* Fix dispatch for `rdiv!` with `LU` (#55764)
* Remove overwritten method of OffsetArray (#56414)
This is overwritten three definitions later in
`Base.reshape(A::OffsetArray, inds::Colon)`.
Should remove warnings I saw when testing a package that uses it.
* Add a missing GC root in constant declaration (#56408)
As pointed out in
https://github.com/JuliaLang/julia/pull/56224#discussion_r1816974147.
* Teach compiler about partitioned bindings (#56299)
This commit teaches to compiler to update its world bounds whenever it
looks at a binding partition, making the compiler sound in the presence
of a partitioned binding. The key adjustment is that the compiler is no
longer allowed to directly query the binding table without recording the
world bounds, so all the various abstract evaluations that look at
bindings need to be adjusted and are no longer pure tfuncs. We used to
look at bindings a lot more, but thanks to earlier prep work to remove
unnecessary binding-dependent code (#55288, #55289 and #55271), these
changes become relatively straightforward.
Note that as before, we do not create any binding partitions by default,
so this commit is mostly preperatory.
---------
Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com>
* Restore JL_NOTSAFEPOINT in jl_stderr_obj (#56407)
This is not a function we're really using, but it's used in the
embedding examples, so I'm sure somebody would complain if I deleted it
or made it a safepoint, so let's just give the same best-effort result
as before.
* reland "Inlining: Remove outdated code path for GlobalRef movement (#46880)" (#56382)
From the description of the original PR:
> We used to not allow `GlobalRef` in `PhiNode` at all (because they
> could have side effects). However, we then change the IR to make
> side-effecting `GlobalRef`s illegal in statement position in general,
> so now `PhiNode`s values are just regular value position, so there's
> no reason any more to try to move `GlobalRef`s out to statement
> position in inlining. Moreover, doing so introduces a bunch of
> unnecessary `GlobalRef`s that weren't being moved back. We could fix
> that separately by setting appropriate flags, but it's simpler to just
> get rid of this special case entirely.
This change itself does not sound to have any issues, and in fact, it is
very useful for keeping the IR slim, especially in code generated by
Cassette-like systems, so I would like to reland it.
However, the original PR was reverted in JuliaLang/julia#46951 due to
bugs like JuliaLang/julia#46940 and JuliaLang/julia#46943. I could not
reproduce these bugs on my end (maybe they have been fixed on some
GC-side fixes?), so I believe relanding the original PR’s changes would
not cause any issues, but it is necessary to confirm that similar
problems do not arise before merging this PR.
* copy effects key to `Base.infer_effects` (#56363)
Copied from the docstring of `Core.Compiler.Effects`, this makes it
easier to figure out what the output of `Base.infer_effects` is actually
telling you.
* Fix `make install` for asan build (#56347)
Now the makescript finds libclang_rt.asan-x86_64.so for example.
The change from `-0` to `-1` is as with `-1`, `libclang_rt.asan-*` is
searched for in `usr/lib/julia` instead of `usr/lib`.
* Add dims check to triangular mul (#56393)
This adds a dimension check to triangular matrix multiplication methods.
While such checks already exist in the individual branches (occasionally
within `BLAS` methods), having these earlier would permit certain
optimizations, as we are assured that the axes are compatible. This
potentially duplicates the checks, but this is unlikely to be a concern
given how cheap the checks are.
I've also reused the `check_A_mul_B!_sizes` function that is defined in
`bidiag.jl`, instead of hard-coding the checks.
Further, I've replaced some hard-coded loop ranges by the corresponding
`axes` and `first/lastindex` calls. These are identical under the
1-based indexing assumption, but the `axes` variants are easier to read
and reason about.
* clarify short-circuit && and || docs (#56420)
This clarifies the docs to explain that `a && b` is equivalent to `a ? b
: false` and that `a || b` is equivalent to `a ? true : b`.
In particular, this explains why the second argument does not need to be
a boolean value, which is a common point of confusion. (See e.g. [this
discourse
thread](https://discourse.julialang.org/t/internals-of-assignment-when-doing-short-circuit-evaluation/122178/2?u=stevengj).)
* docs: replace 'leaf types' with 'concrete types' (#56418)
Fixes #55044
---------
Co-authored-by: inkydragon <inkydragon@users.noreply.github.com>
* Remove aggressive constprop annotation on generic_matmatmul_wrapper! (#56400)
This annotation seems unnecessary, as the method gets inlined and
there's no computation being carried out using the value of the
constant.
* Clarify the FieldError docstring (#55222)
* Allow `Time`s to be rounded to `Period`s (#52629)
Co-authored-by: CyHan <git@wo-class.cn>
Co-authored-by: Curtis Vogt <curtis.vogt@gmail.com>
* Replace unconditional store with cmpswap to avoid deadlocking in jl_fptr_wait_for_compiled_addr (#56444)
That unconditional store could overwrite the actual compiled code in
that pointer, so make it a cmpswap
* Correct nothrow modeling of `get_binding_type` (#56430)
As pointed out in
https://github.com/JuliaLang/julia/pull/56299#discussion_r1826509185,
although the bug predates that PR.
* add tip for module docstrings before load (#56445)
* compiler: Strengthen some assertions and fix a couple small bugs (#56449)
* inference: minor follow-ups to JuliaLang/julia#56299 (#56450)
* Ensure that String(::Memory) returns only a String, not any owner (#56438)
Fixes #56435
* Take safepoint lock before going to sleep in the scheduler. (#56443)
This avoids a deadlock during exit. Between a thread going to sleep and
the thread exiting.
* Profile: mention `kill -s SIGUSR1 julia_pid` for Linux (#56441)
currentlu this route is mentioned in docs
https://docs.julialang.org/en/v1/stdlib/Profile/#Triggered-During-Execution
but missing from the module docstring, this should help users who have
little idea how to "send a kernel signal to a process" to get started
---------
Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
* Fix and test an overflow issue in `searchsorted` (#56464)
And remove `searchsorted` special cases for offset arrays in tests that
had the impact of bypassing actually testing `searchsorted` behavior on
offset arrays
To be clear, after this bugfix the function is still broken, just a little bit less so.
* Update docs of calling convention arg in `:foreigncall` AST node (#56417)
* `step(::AbstractUnitRange{Bool})` should return `Bool` (#56405)
The issue was introduced by #27302 , as
```julia
julia> true-false
1
```
By definitions below, `AbstractUnitRange{Bool} <: OrdinalRange{Bool,
Bool}` whose step type is `Bool`.
https://github.com/JuliaLang/julia/blob/da74ef1933b12410b217748e0f7fbcbe52e10d29/base/range.jl#L280-L299
---------
Co-authored-by: Matt Bauman <mbauman@gmail.com>
Co-authored-by: Matt Bauman <mbauman@juliahub.com>
* fixup! JuliaLang/julia#56028, fix up the type-level escapability check
In JuliaLang/julia#56028, the type-level escapability check was changed
to use `is_mutation_free_argtype`, but this was a mistake because EA no
longer runs for structs like
`mutable struct ForeignBuffer{T}; const ptr::Ptr{T}; end`.
This commit changes it to use `is_identity_free_argtype` instead, which
can be used to detect whether a type may contain any mutable allocations
or not.
* add `show(::IO, ::ArgEscapeInfo)`
* EA: disable finalizer inlining for allocations that are edges of `PhiNode`s (#56455)
The current EA-based finalizer inlining implementation can create
invalid IR when the target object is later aliased as a `PhiNode`, which
was causing #56422.
In such cases, finalizer inlining for the allocations that are edges of
each `PhiNode` should be avoided, and instead, finalizer inlining should
ideally be applied to the `PhiNode` itself, but implementing that is
somewhat complex. As a temporary fix, this commit disables inlining in
those cases.
- fixes #56422
* make `verify_ir` error messages more informative (#56452)
Currently, when `verify_ir` finds an error, the `IRCode` is printed, but
it's not easy to determine which method instance generated that
`IRCode`. This commit adds method instance and code location information
to the error message, making it easier to identify the problematic code.
E.g.:
```julia
[...]
610 │ %95 = builtin Core.tuple(%48, %94)::Tuple{GMT.Gdal.IGeometry, GMT.Gdal.IGeometry}
└─── return %95
ERROR: IR verification failed.
Code location: ~/julia/packages/GMT/src/gdal_extensions.jl:606
Method instance: MethodInstance for GMT.Gdal.helper_2geoms(::Matrix{Float64}, ::Matrix{Float64})
Stacktrace:
[1] error(::String, ::String, ::String, ::Symbol, ::String, ::Int32, ::String, ::String, ::Core.MethodInstance)
@ Core.Compiler ./error.jl:53
[...]
```
* [GHA] Explicitly install Julia for whitespace workflow (#56468)
So far we relied on the fact that Julia comes in the default Ubuntu
images on GitHub Actions runners, but this may change in the future
(although there's apparently no plan in this direction for the time
being). To make the workflow more future-proof, we now explicitly
install Julia using a dedicated workflow.
* Allow taking Matrix slices without an extra allocation (#56236)
Since changing Array to use Memory as the backing, we had the option of
making non-Vector arrays more flexible, but had instead preserved the
restriction that they must be zero offset and equal in length to the
Memory. This results in extra complexity, restrictions, and allocations
however, but doesn't gain many known benefits. Nanosoldier shows a
decrease in performance on linear eachindex loops, which we theorize is
due to a minor failure to CSE before SCEV or a lack of NUW/NSW on the
length multiplication calculation.
* [late-gc-lowering] null-out GC frame slots for dead objects (#52935)
Should fix https://github.com/JuliaLang/julia/issues/51818.
MWE:
```julia
function testme()
X = @noinline rand(1_000_000_00)
Y = @noinline sum(X)
X = nothing
GC.gc()
return Y
end
```
Note that it now stores a `NULL` in the GC frame before calling
`jl_gc_collect`.
Before:
```llvm
; Function Signature: testme()
; @ /Users/dnetto/Personal/test.jl:3 within `testme`
define double @julia_testme_535() #0 {
top:
%gcframe1 = alloca [3 x ptr], align 16
call void @llvm.memset.p0.i64(ptr align 16 %gcframe1, i8 0, i64 24, i1 true)
%pgcstack = call ptr inttoptr (i64 6595051180 to ptr)(i64 262) #10
store i64 4, ptr %gcframe1, align 16
%task.gcstack = load ptr, ptr %pgcstack, align 8
%frame.prev = getelementptr inbounds ptr, ptr %gcframe1, i64 1
store ptr %task.gcstack, ptr %frame.prev, align 8
store ptr %gcframe1, ptr %pgcstack, align 8
; @ /Users/dnetto/Personal/test.jl:4 within `testme`
%0 = call nonnull ptr @j_rand_539(i64 signext 100000000)
%gc_slot_addr_0 = getelementptr inbounds ptr, ptr %gcframe1, i64 2
store ptr %0, ptr %gc_slot_addr_0, align 16
; @ /Users/dnetto/Personal/test.jl:5 within `testme`
%1 = call double @j_sum_541(ptr nonnull %0)
; @ /Users/dnetto/Personal/test.jl:7 within `testme`
; ┌ @ gcutils.jl:132 within `gc` @ gcutils.jl:132
call void @jlplt_ijl_gc_collect_543_got.jit(i32 1)
%frame.prev4 = load ptr, ptr %frame.prev, align 8
store ptr %frame.prev4, ptr %pgcstack, align 8
; └
; @ /Users/dnetto/Personal/test.jl:8 within `testme`
ret double %1
}
```
After:
```llvm
; Function Signature: testme()
; @ /Users/dnetto/Personal/test.jl:3 within `testme`
define double @julia_testme_752() #0 {
top:
%gcframe1 = alloca [3 x ptr], align 16
call void @llvm.memset.p0.i64(ptr align 16 %gcframe1, i8 0, i64 24, i1 true)
%pgcstack = call ptr inttoptr (i64 6595051180 to ptr)(i64 262) #10
store i64 4, ptr %gcframe1, align 16
%task.gcstack = load ptr, ptr %pgcstack, align 8
%frame.prev = getelementptr inbounds ptr, ptr %gcframe1, i64 1
store ptr %task.gcstack, ptr %frame.prev, align 8
store ptr %gcframe1, ptr %pgcstack, align 8
; @ /Users/dnetto/Personal/test.jl:4 within `testme`
%0 = call nonnull ptr @j_rand_756(i64 signext 100000000)
%gc_slot_addr_0 = getelementptr inbounds ptr, ptr %gcframe1, i64 2
store ptr %0, ptr %gc_slot_addr_0, align 16
; @ /Users/dnetto/Personal/test.jl:5 within `testme`
%1 = call double @j_sum_758(ptr nonnull %0)
store ptr null, ptr %gc_slot_addr_0, align 16
; @ /Users/dnetto/Personal/test.jl:7 within `testme`
; ┌ @ gcutils.jl:132 within `gc` @ gcutils.jl:132
call void @jlplt_ijl_gc_collect_760_got.jit(i32 1)
%frame.prev6 = load ptr, ptr %frame.prev, align 8
store ptr %frame.prev6, ptr %pgcstack, align 8
; └
; @ /Users/dnetto/Personal/test.jl:8 within `testme`
ret double %1
}
```
* Added test for resolving array references in exprresolve (#56471)
added test to take care of non-real-index handling while resolving array
references in exprresolve to test julia/base/cartesian.jl - line 427 to
432
* Fix and test searchsorted for arrays whose first index is `typemin(Int)` (#56474)
This fixes the issue reported in
https://github.com/JuliaLang/julia/issues/56457#issuecomment-2457223264
which, combined with #56464 which fixed the issue in the OP, fixes #56457.
`searchsortedfirst` was fine all along, but I added it to tests regardless.
* Move Core.Compiler into Base
This is the first step in what I am hoping will eventually result in making
the compiler itself and upgradable stdlib. Over time, we've gained several
non-Base consumers of `Core.Compiler`, and we've reached a bit of a breaking
point where maintaining those downstream dependencies is getting more difficult
than the close coupling of Core.Compiler to the runtime is worth.
In this first step, I am moving Core.Compiler into Base, ending the duplication
of common data structure and generic functions between Core.Compiler and Base.
This split goes back quite far (although not all the way) to the early days of
Julia and predates the world-age mechanism.
The extant Base and Core.Compiler environments have some differences
(other than the duplication). I think the primary ones are (but I will add
more here if somebody points one out).
- `Core.Compiler` does not use `getproperty`
- `Core.Compiler` does not have extensible `==` equality
In this, I decided to retain the former by setting `getproperty = getfield`
for Core.Compiler itself (though of course not for the datatstructures shared
with Base). I don't think it's strictly necessary, but might as well.
For equality, I decided the easiest thing to do would be to try to merge
the equalities and see what happens. In general, Core.Compiler is relatively
restricted in the kinds of equality comparisons it can make, so I think it'll
work out fine, but we can revisit this.
This seems to be fully working and most of this is just moving code around.
I think most of that refactoring is independently useful, so I'll pull some
of it out into separate PRs to make this PR more manageable.
* Delete buggy `stat(::Integer)` method (#54855)
"Where did someone get a RawFD as an integer anyway?" -@stefankarpinski
See also #51711
Fixes #51710
* missing gc-root store in subtype (#56472)
Fixes #56141
Introduced by #52228 (a624d445c02c)
* further defer jl_insert_backedges after loading (#56447)
Finish fully breaking the dependency between method insertions and
inferring whether the cache is valid. The cache should be inferable in
parallel and in aggregate after all loading is finished. This prepares
us for moving this code into Julia (Core.Compiler) next.
* count bytes allocated through malloc more precisely (#55223)
Should make the accounting for memory allocated through malloc a bit
more accurate.
Should also simplify the accounting code by eliminating the use of
`jl_gc_count_freed` in `jl_genericmemory_to_string`.
* Fix external IO loop thead interaction and add function to Base.Experimental to facilitate it's use. Also add a test. (#55529)
While looking at https://github.com/JuliaLang/julia/issues/55525 I found
that the implementation wasn't working correctly.
I added it to Base.Experimental so people don't need to handroll their
own and am also testing a version of what the issue was hitting.
* [REPL] raise default implicit `show` limit to 1MiB (#56297)
https://github.com/JuliaLang/julia/pull/53959#issuecomment-2426946640
I would like to understand more where these issues are coming from; it
would be easy to exempt some types from Base or Core with
```julia
REPL.show_limited(io::IO, mime::MIME, x::SomeType) = show(io, mime, x)
```
but I'm not sure which are causing problems in practice.
But meanwhile I think raising the limit makes sense.
* Add a docstring for `Base.divgcd` (#53769)
Co-authored-by: Sukera <11753998+Seelengrab@users.noreply.github.com>
* Fix compilation warning on aarch64-linux (#56480)
This fixes the warning:
```
/cache/build/default-aws-aarch64-ci-1-3/julialang/julia-master/src/stackwalk.c: In function 'jl_simulate_longjmp':
/cache/build/default-aws-aarch64-ci-1-3/julialang/julia-master/src/stackwalk.c:995:22: warning: initialization of 'mcontext_t *' {aka 'struct sigcontext *'} from incompatible pointer type 'struct unw_sigcontext *' [-Wincompatible-pointer-types]
995 | mcontext_t *mc = &c->uc_mcontext;
| ^
```
This is the last remaining warning during compilation on aarch64-linux.
* Make Compiler an independent package
This is a further extension to #56128 to make the compiler into a proper
independent, useable outside of `Base` as `using Compiler` in the same way
that `JuliaSyntax` works already. InteractiveUtils gains a new `@activate`
macro that can be used to activate an outside Compiler package, either for
reflection only or for codegen also.
* Make heap size hint available as an env variable (#55631)
This makes `JULIA_HEAP_SIZE_HINT` the environment variable version of
the `--heap-size-hint` command-line flag. Seems like there was interest
in
https://github.com/JuliaLang/julia/pull/45369#issuecomment-1544204022.
The same syntax is used as for the command-line version with, for
example, `2G` => 2 GB and `200M` => 200 MB.
@oscardssmith want to take a look?
* Allow indexing `UniformScaling` with `CartesianIndex{2}` (#56461)
Since indexing with two `Integer`s is defined, we might as well define
indexing with a `CartesianIndex`. This makes certain loops convenient
where the index is obtained using `eachindex`.
* Simplify first index in `FastContiguousSubArray` definition (#56491)
Since `Slice <: AbstractUnitRange` and `Union{Slice, AbstractUnitRange}
== AbstractUnitRange`, we may simplify the first index.
* Make `popat!` support `@inbounds` (#56323)
Co-authored-by: Jishnu Bhattacharya <jishnub.github@gmail.com>
* NEWS.md: clarify `--trim` (#56460)
Co-authored-by: Matt Bauman <mbauman@gmail.com>
* Remove aggressive constprop annotation from 2x2 and 3x3 matmul (#56453)
Removing these annotations reduces ttfx slightly.
```julia
julia> using LinearAlgebra
julia> A = rand(2,2);
julia> @time mul!(similar(A), A, A, 1, 2);
0.296096 seconds (903.49 k allocations: 44.313 MiB, 4.25% gc time, 99.98% compilation time) # nightly
0.286009 seconds (835.88 k allocations: 40.732 MiB, 3.29% gc time, 99.98% compilation time) # this PR
```
* `sincos` for non-float symmetric matrices (#56484)
Ensures that the `eltype` of the array to which the result of `sincos`
is a floating-point one, even if the argument doesn't have a
floating-point `eltype`.
After this, the following works:
```julia
julia> A = diagm(0=>1:3)
3×3 Matrix{Int64}:
1 0 0
0 2 0
0 0 3
julia> sincos(A)
([0.8414709848078965 0.0 0.0; 0.0 0.9092974268256817 0.0; 0.0 0.0 0.1411200080598672], [0.5403023058681398 0.0 0.0; 0.0 -0.4161468365471424 0.0; 0.0 0.0 -0.9899924966004454])
```
* Specialize 2-arg `show` for `LinearIndices` (#56482)
After this,
```julia
julia> l = LinearIndices((1:3, 1:4));
julia> show(l)
LinearIndices((1:3, 1:4))
```
The printed form is a valid constructor.
* Avoid constprop in `syevd!` and `syev!` (#56442)
This improves compilation times slightly:
```julia
julia> using LinearAlgebra
julia> A = rand(2,2);
julia> @time eigen!(Hermitian(A));
0.163380 seconds (180.51 k allocations: 8.760 MiB, 99.88% compilation time) # master
0.155285 seconds (163.77 k allocations: 7.971 MiB, 99.87% compilation time) # This PR
```
The idea is that the constant propagation is only required to infer the
return type, and isn't necessary in the body of the method. We may
therefore annotate the body with a `@constprop :none`.
* make: define `basecompiler.ji` target (#56498)
For easier experimentation with just the bootstrap process.
Additionally, as a follow-up to JuliaLang/julia#56409, this commit also
includes some minor cosmetic changes.
* speed up bootstrapping by compiling few optimizer subroutines earlier (#56501)
Speeds up the bootstrapping process by about 30 seconds.
* remove top-level branches checking for Base (#56507)
These are no longer needed, now that the files are no longer included
twice.
* Undo the decision to publish incomplete types to the binding table (#56497)
This effectively reverts #36121 and replaces it with #36111, which was
the originally proposed alternative to fix #36104. To recap, the
question is what should happen for
```
module Foo
struct F
v::Foo.F
end
end
```
i.e. where the type reference tries to refer to the newly defined type
via its global path. In #36121 we adjusted things so that we first
assign the type to its global binding and then evaluate the field type
(leaving the type in an incomplete state in the meantime). The primary
reason that this choice was that we would have to deal with incomplete
types assigned to global bindings anyway if we ever did #32658. However,
I think this was the wrong choice. There is a difference between
allowing incomplete types and semantically forcing incomplete types to
be globally observable every time a new type is defined.
The situation was a little different four years ago, but with more
extensive threading (which can observe the incompletely constructed
type) and the upcoming completion of bindings partition, the situation
is different. For bindings partition in particular, this would require
two invalidations on re-definition, one to the new incomplete type and
then back to the complete type. I don't think this is worth it, for the
(somewhat niche and possibly-should-be- deprecated-future) case of
refering to incompletely defined types by their global names.
So let's instead try the hack in #36111, which does a frontend rewrite
of the global path. This should be sufficient to at least address the
obvious cases.
* Merge identical methods for Symmetric/Hermitian and SymTridiagonal (#56434)
Since the methods do identical things, we may define each method once
for a union of types instead of defining methods for each type.
* Specialize findlast for integer AbstractUnitRanges and StepRanges (#54902)
For monotonic ranges, `findfirst` and `findlast` with `==(val)` as the
predicate should be identical, as each value appears only once in the
range. Since `findfirst` is specialized for some ranges, we may define
`findlast` as well analogously.
On v"1.12.0-DEV.770"
```julia
julia> @btime findlast(==(1), $(Ref(1:1_000))[])
1.186 μs (0 allocations: 0 bytes)
1
```
This PR
```julia
julia> @btime findlast(==(1), $(Ref(1:1_000))[])
3.171 ns (0 allocations: 0 bytes)
1
```
I've also specialized `findfirst(iszero, r::AbstractRange)` to make this
be equivalent to `findfirst(==(0), ::AbstractRange)` for numerical
ranges. Similarly, for `isone`. These now take the fast path as well.
Thirdly, I've added some `convert` calls to address issues like
```julia
julia> r = Int128(1):Int128(1):Int128(4);
julia> findfirst(==(Int128(2)), r) |> typeof
Int128
julia> keytype(r)
Int64
```
This PR ensures that the return type always corresponds to `keytype`,
which is what the docstring promises.
This PR also fixes
```julia
julia> findfirst(==(0), UnitRange(-0.5, 0.5))
ERROR: InexactError: Int64(0.5)
Stacktrace:
[1] Int64
@ ./float.jl:994 [inlined]
[2] findfirst(p::Base.Fix2{typeof(==), Int64}, r::UnitRange{Float64})
@ Base ./array.jl:2397
[3] top-level scope
@ REPL[1]:1
```
which now returns `nothing`, as expected.
* Loop over `Iterators.rest` in `_foldl_impl` (#56492)
For reasons that I don't understand, this improves performance in
`mapreduce` in the following example:
```julia
julia> function g(A)
for col in axes(A,2)
mapreduce(iszero, &, view(A, UnitRange(axes(A,1)), col), init=true) || return false
end
return true
end
g (generic function with 2 methods)
julia> A = zeros(2, 10000);
julia> @btime g($A);
28.021 μs (0 allocations: 0 bytes) # nightly v"1.12.0-DEV.1571"
12.462 μs (0 allocations: 0 bytes) # this PR
julia> A = zeros(1000,1000);
julia> @btime g($A);
372.080 μs (0 allocations: 0 bytes) # nightly
321.753 μs (0 allocations: 0 bytes) # this PR
```
It would be good to understand what the underlying issue is, as the two
seem equivalent to me. Perhaps this form makes it clear that it's not,
in fact, an infinite loop?
* better error message for rpad/lpad with zero-width padding (#56488)
Closes #45339 — throw a more informative `ArgumentError` message from
`rpad` and `lpad` if a zero-`textwidth` padding is passed (not a
`DivideError`).
If the padding character has `ncodeunits == 1`, suggests that maybe they
want `str * pad^max(0, npad - ncodeunits(str))` instead.
* Safer indexing in dense linalg methods (#56451)
Ensure that `eachindex` is used consistently alongside `@inbounds`, and
use `diagind` to obtain indices along a diagonal.
* The `info` in LAPACK calls should be a Ref instead of a Ptr (#56511)
Co-authored-by: Viral B. Shah <ViralBShah@users.noreply.github.com>
* Scaling loop instead of broadcasting in strided matrix exp (#56463)
Firstly, this is easier to read. Secondly, this merges the two loops
into one. Thirdly, this avoids the broadcasting latency.
```julia
julia> using LinearAlgebra
julia> A = rand(2,2);
julia> @time LinearAlgebra.exp!(A);
0.952597 seconds (2.35 M allocations: 116.574 MiB, 2.67% gc time, 99.01% compilation time) # master
0.877404 seconds (2.17 M allocations: 106.293 MiB, 2.65% gc time, 99.99% compilation time) # this PR
```
The performance also improves as there are fewer allocations in the
first branch (`opnorm(A, 1) <= 2.1`):
```julia
julia> B = diagm(0=>im.*(float.(1:200))./200, 1=>(1:199)./400, -1=>(1:199)./400);
julia> opnorm(B,1)
1.9875
julia> @btime exp($B);
5.066 ms (30 allocations: 4.89 MiB) # nightly v"1.12.0-DEV.1581"
4.926 ms (27 allocations: 4.28 MiB) # this PR
```
* codegen: Respect binding partition (#56494)
Minor changes to make codegen correct in the face of partitioned
constant bindings. Does not yet handle the envisioned semantics for
globals that change restriction type, which will require a fair bit of
additional work.
* Profile: fix Compiler short path (#56515)
* Check `isdiag` in dense trig functions (#56483)
This improves performance for dense diagonal matrices, as we may apply
the function only to the diagonal elements.
```julia
julia> A = diagm(0=>rand(100));
julia> @btime cos($A);
349.211 μs (22 allocations: 401.58 KiB) # nightly v"1.12.0-DEV.1571"
16.215 μs (7 allocations: 80.02 KiB) # this PR
```
---------
Co-authored-by: Daniel Karrasch <daniel.karrasch@posteo.de>
* Profile: add helper method for printing profile report to file (#56505)
The IOContext part is isn't obvious, because otherwise the IO is assumed
to be 80 chars wide, which makes for bad reports.
* Change in-place exp to out-of-place in matrix trig functions (#56242)
This makes the functions work for arbitrary matrix types that support
`exp`, but not necessarily the in-place `exp!`. For example, the
following works after this:
```julia
julia> m = SMatrix{2,2}(1:4);
julia> cos(m)
2×2 SMatrix{2, 2, Float64, 4} with indices SOneTo(2)×SOneTo(2):
0.855423 -0.166315
-0.110876 0.689109
```
There's a slight performance improvement as well because we don't
compute `im*A` and `-im*A` separately, but we negate the first to obtain
the second.
```julia
julia> A = rand(ComplexF64,100,100);
julia> @btime sin($A);
2.796 ms (48 allocations: 1.84 MiB) # nightly v"1.12.0-DEV.1571"
2.304 ms (48 allocations: 1.84 MiB) # this PR
```
* Test: Don't change scope kind in `test_{warn,nowarn}` (#56524)
This was part of #56509, but is an independent bugfix. The basic issue
is that these macro were using `do` block internally. This is
undesirable for test macros, because we would like them not to affect
the behavior of what they're testing. E.g. right now:
```
julia> using Test
julia> const x = 1
1
julia> @test_nowarn const x = 1
ERROR: syntax: `global const` declaration not allowed inside function around /home/keno/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:927
Stacktrace:
[1] top-level scope
@ REPL[3]:1
```
This PR just writes out the try/finally manually, so the above works
fine after this PR.
* For loop instead of while in generic `copyto!` (#56517)
This appears to improve performance.
```julia
j…
vtjnash
added a commit
that referenced
this pull request
Dec 9, 2024
While doing some work on analyzing what code runs at toplevel, I found a few things to simplify or fix: - simplify float.jl loop not to call functions just defined to get back the value just stored there - add method to the correct checkbounds function (instead of a local) - missing world push/pop around jl_interpret_toplevel_thunk after #56509 - jl_lineno use maybe missed in #53799 - add some debugging dump for scm_to_julia_ mistakes
felipenoris
pushed a commit
to felipenoris/OptimizingIR.jl
that referenced
this pull request
Dec 22, 2024
Julia 1.12 will be stricter about when world age gets incremented implicitly in toplevel scope (see JuliaLang/julia#56509). This PR adds explicit world age increments after compiling code using the Native target, which will likely be required on Julia 1.12 (and are harmless on prior julia versions).
stevengj
pushed a commit
that referenced
this pull request
Jan 2, 2025
While doing some work on analyzing what code runs at toplevel, I found a few things to simplify or fix: - simplify float.jl loop not to call functions just defined to get back the value just stored there - add method to the correct checkbounds function (instead of a local) - missing world push/pop around jl_interpret_toplevel_thunk after #56509 - jl_lineno use maybe missed in #53799 - add some debugging dump for scm_to_julia_ mistakes
This was referenced Jan 8, 2025
udesou
added a commit
to mmtk/julia
that referenced
this pull request
Jan 31, 2025
* Make a failed extension load throw an error during pre-compilation (#56668)
Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
* Remove arraylist_t from external native code APIs. (#56693)
This makes them usable for external consumers like GPUCompiler.jl.
* fix world_age_at_entry in codegen (#56700)
* Fix typo in `@cmd` docstring (#56664)
I'm not sure what `` `cmd` `` could refer to, but it would make sense to
refer to `` `str` `` in this case. I'm assuming it's a typo.
* deps/pcre: Update to version 10.44 (#56704)
* compile: make more efficient by discarding internal names (#56702)
These are not user-visible, so this makes the compiler faster and more
efficient with no effort on our part, and avoids duplicating the
debug_level parameter.
* Automatically enable JITPROFILING with ITTAPI (#55598)
This helps when profiling remotely since VTunes doesn't support
setting environment variables on remote systems.
Will still respect `ENABLE_JITPROFILING=0`.
* Fix string handling in jlchecksum (#56720)
A `TAGGED_RELEASE_BANNER` with spaces such as `Official
https://julialang.org release` produces the error
`/cache/build/builder-amdci4-5/julialang/julia-master/deps/tools/jlchecksum:
66: [: Official: unexpected operator`.
* Clarifying ispunct behavior difference between Julia and C in documentation (#56727)
Fixes #56680. This PR updates the documentation for the ispunct function
in Julia to explicitly note its differing behavior from the similarly
named function in C.
---------
Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com>
* [NEWS.md] Add PR numbers and remove some 1.11 changes that accidentally came back. (#56722)
* ircode: cleanup code crud
- support gc running
- don't duplicate field 4
- remove some unused code only previously needed for handling cycles
(which are not valid in IR)
* ircode: small optimization for nearby ssavalue
Since most ssavalue are used just after their def, this gives a small
memory savings on compressed IR (a fraction of a percent).
* Fix scope of hoisted signature-local variables (#56712)
When we declare inner methods, e.g. the `f` in
```
function fs()
f(lhs::Integer) = 1
f(lhs::Integer, rhs::(local x=Integer; x)) = 2
return f
end
```
we must hoist the definition of the (appropriately mangled) generic
function `f` to top-level, including all variables that were used in the
signature definition of `f`. This situation is a bit unique in the
language because it uses inner function scope, but gets executed in
toplevel scope. For example, you're not allowed to use a local of the
inner function in the signature definition:
```
julia> function fs()
local x=Integer
f(lhs::Integer, rhs::x) = 2
return f
end
ERROR: syntax: local variable x cannot be used in closure declaration
Stacktrace:
[1] top-level scope
@ REPL[3]:1
```
In particular, the restriction is signature-local:
```
julia> function fs()
f(rhs::(local x=Integer; x)) = 1
f(lhs::Integer, rhs::x) = 2
return f
end
ERROR: syntax: local variable x cannot be used in closure declaration
Stacktrace:
[1] top-level scope
@ REPL[4]:1
```
There's a special intermediate form `moved-local` that gets generated
for this definition. In c6c3d72d1cbddb3d27e0df0e739bb27dd709a413, this
form stopped getting generated for certain inner methods. I suspect this
happened because of the incorrect assumption that the set of moved
locals is being computed over all signatures, rather than being a
per-signature property.
The result of all of this was that this is one of the few places where
lowering still generated a symbol as the lhs of an assignment for a
global (instead of globalref), because the code that generates the
assignment assumes it's a local, but the later pass doesn't know this.
Because we still retain the code for this from before we started using
globalref consistently, this wasn't generally causing a problems, except
possibly leaking a global (or potentially assigning to a global when
this wasn't intended). However, in follow on work, I want to make use of
knowing whether the LHS is a global or local in lowering, so this was
causing me trouble.
Fix all of this by putting back the `moved-local` where it was dropped.
Fixes #56711
* ircode: avoid serializing ssaflags in the common case when they are all zero
When not all-zero, run-length encoding would also probably be great
here for lowered code (before inference).
* Extend `invoke` to accept CodeInstance (#56660)
This is an alternative mechanism to #56650 that largely achieves the
same result, but by hooking into `invoke` rather than a generated
function. They are orthogonal mechanisms, and its possible we want both.
However, in #56650, both Jameson and Valentin were skeptical of the
generated function signature bottleneck. This PR is sort of a hybrid of
mechanism in #52964 and what I proposed in
https://github.com/JuliaLang/julia/pull/56650#issuecomment-2493800877.
In particular, this PR:
1. Extends `invoke` to support a CodeInstance in place of its usual
`types` argument.
2. Adds a new `typeinf` optimized generic. The semantics of this
optimized generic allow the compiler to instead call a companion
`typeinf_edge` function, allowing a mid-inference interpreter switch
(like #52964), without being forced through a concrete signature
bottleneck. However, if calling `typeinf_edge` does not work (e.g.
because the compiler version is mismatched), this still has well defined
semantics, you just don't get inference support.
The additional benefit of the `typeinf` optimized generic is that it
lets custom cache owners tell the runtime how to "cure" code instances
that have lost their native code. Currently the runtime only knows how
to do that for `owner == nothing` CodeInstances (by re-running
inference). This extension is not implemented, but the idea is that the
runtime would be permitted to call the `typeinf` optimized generic on
the dead CodeInstance's `owner` and `def` fields to obtain a cured
CodeInstance (or a user-actionable error from the plugin).
This PR includes an implementation of `with_new_compiler` from #56650.
This PR includes just enough compiler support to make the compiler
optimize this to the same code that #56650 produced:
```
julia> @code_typed with_new_compiler(sin, 1.0)
CodeInfo(
1 ─ $(Expr(:foreigncall, :(:jl_get_tls_world_age), UInt64, svec(), 0, :(:ccall)))::UInt64
│ %2 = builtin Core.getfield(args, 1)::Float64
│ %3 = invoke sin(%2::Float64)::Float64
└── return %3
) => Float64
```
However, the implementation here is extremely incomplete. I'm putting it
up only as a directional sketch to see if people prefer it over #56650.
If so, I would prepare a cleaned up version of this PR that has the
optimized generics as well as the curing support, but not the full
inference integration (which needs a fair bit more work).
* Update references to LTS from v1.6 to v1.10 (#56729)
* lowering: Canonicalize to builtins for global assignment (#56713)
This adjusts lowering to emit `setglobal!` for assignment to globals,
thus making the `=` expr head used only for slots in `CodeInfo` and
entirely absent in `IRCode`. The primary reason for this is just to
reduce the number of special cases that compiler passes have to reason
about. In IRCode, `=` was already essentially equivalent to
`setglobal!`, so there's no good reason not to canonicalize.
Finally, the `=` syntax form for globals already gets recognized
specially to insert `convert` calls to their declared binding type, so
this doesn't impose any additional requirements on lowering to
distinguish local from global assignments. In general, I'd also like to
separate syntax and intermediate forms as much as possible where their
semantics differ, which this accomplises by just using the builtin.
This change is mostly semantically invisible, except that spliced-in
GlobalRefs now declare their binding because they are indistinguishable
from ordinary assignments at the stage where I inserted the lowering. If
we want to, we can preserve the difference, but it'd be a bit more
annoying for not much benefit, because:
1. The spliced in version was only recently made to work anyway, and
2. The semantics of when exactly bindings are declared is still messy on
master anyway and will get tweaked shortly in further binding partitions
work.
* Actually show glyphs for latex or emoji shortcodes being suggested in the REPL (#54800)
When a user requests a completion for a backslash shortcode, this PR
adds the glyphs for all the suggestions to the output. This makes it
much easier to find the result one is looking for, especially if the
user doesn't know all latex and emoji specifiers by heart.
Before:
<img width="813" alt="image"
src="https://github.com/JuliaLang/julia/assets/22495855/bf651399-85a6-4677-abdc-c66a104e3b89">
After:
<img width="977" alt="image"
src="https://github.com/JuliaLang/julia/assets/22495855/04c53ea2-318f-4888-96eb-0215b49c10f3">
---------
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* Update annotated.jl docstrings according to #55741 (#56736)
Annotations now use a NamedTuple
* effects: pack bits better (#56737)
There is no reason to preserve duplicates of the bits for the value
before and after inference, and many of the numbers in the comments had
gotten incorrect. Now we are able to pack all 16 bits of currently
defined bitflags into 20 bits, instead of 25 bits (although either case
still rounds up to 32).
There was also no reason for InferenceState to be mutating of CodeInfo
during execution, so remove that mutation.
* 🤖 [master] Bump the Pkg stdlib from 7b759d7f0 to d84a1a38b (#56743)
Stdlib: Pkg
URL: https://github.com/JuliaLang/Pkg.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 7b759d7f0
New commit: d84a1a38b
Julia version: 1.12.0-DEV
Pkg version: 1.12.0
Bump invoked by: @KristofferC
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/Pkg.jl/compare/7b759d7f0af56c5ad01f2289bbad71284a556970...d84a1a38b6466fa7400e9ad2874a0ef963a10456
```
$ git log --oneline 7b759d7f0..d84a1a38b
d84a1a38b Allow use of a url and subdir in [sources] (#4039)
cd75456a8 Fix heading (#4102)
b61066120 rename FORMER_STDLIBS -> UPGRADABLE_STDLIBS (#4070)
814949ed2 Increase version of `StaticArrays` in `why` tests (#4077)
83e13631e Run CI on backport branch too (#4094)
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* Hide IRShow include from Revise (#56756)
Revise in theory wants to re-evaluate this include, but it fails at
doing so, because the include call no longer works after bootstrap. It
happens to work right now on master, because the lowering of
`Compiler.include` happens to hide the include call from Revise, but
that's a Revise bug I'm about to fix. Address this by moving the include
call into the package and using an absolute include if necessary.
* fixup!: JuliaLang/julia#56756 (#56758)
We need to quote it, otherwise it would result in `UnderVarError`.
* Accept more general Integer sizes in reshape (#55521)
This PR generalizes the `reshape` methods to accept `Integer`s instead
of `Int`s, and adds a `_reshape_uncolon` method for `Integer` arguments.
The current `_reshape_uncolon` method that accepts `Int`s is left
unchanged to ensure that the inferred types are not impacted. I've also
tried to ensure that most `Integer` subtypes in `Base` that may be
safely converted to `Int`s pass through that method.
The call sequence would now go like this:
```julia
reshape(A, ::Tuple{Vararg{Union{Integer, Colon}}}) -> reshape(A, ::Tuple{Vararg{Integer}}) -> reshape(A, ::Tuple{Vararg{Int}}) (fallback)
```
This lets packages define `reshape(A::CustomArray, ::Tuple{Integer,
Vararg{Integer}})` without having to implement `_reshape_uncolon` by
themselves (or having to call internal `Base` functions, as in
https://github.com/JuliaArrays/FillArrays.jl/issues/373). `reshape`
calls involving a `Colon` would convert this to an `Integer` in `Base`,
and then pass the `Integer` sizes to the custom method defined in the
package.
This PR does not resolve issues like
https://github.com/JuliaLang/julia/issues/40076 because this still
converts `Integer`s to `Int`s in the actual reshaping step. However,
`BigInt` sizes that may be converted to `Int`s will work now:
```julia
julia> reshape(1:4, big(2), big(2))
2×2 reshape(::UnitRange{Int64}, 2, 2) with eltype Int64:
1 3
2 4
julia> reshape(1:4, big(1), :)
1×4 reshape(::UnitRange{Int64}, 1, 4) with eltype Int64:
1 2 3 4
```
Note that the reshape method with `Integer` sizes explicitly converts
these to `Int`s to avoid self-recursion (as opposed to calling
`to_shape` to carry out the conversion implicitly). In the future, we
may want to decide what to do with types or values that can't be
converted to an `Int`.
---------
Co-authored-by: Neven Sajko <s@purelymail.com>
* drop llvm 16 support (#56751)
Co-authored-by: Valentin Churavy <v.churavy@gmail.com>
* Address some post-commit review from #56660 (#56747)
Some more questions still to be answered, but this should address the
immediate actionable review items.
* Add per-task metrics (#56320)
Close https://github.com/JuliaLang/julia/issues/47351 (builds on top of
https://github.com/JuliaLang/julia/pull/48416)
Adds two per-task metrics:
- running time = amount of time the task was actually running (according
to our scheduler). Note: currently inclusive of GC time, but would be
good to be able to separate that out (in a future PR)
- wall time = amount of time between the scheduler becoming aware of
this task and the task entering a terminal state (i.e. done or failed).
We record running time in `wait()`, where the scheduler stops running
the task as well as in `yield(t)`, `yieldto(t)` and `throwto(t)`, which
bypass the scheduler. Other places where a task stops running (for
`Channel`, `ReentrantLock`, `Event`, `Timer` and `Semaphore` are all
implemented in terms of `wait(Condition)`, which in turn calls `wait()`.
`LibuvStream` similarly calls `wait()`.
This should capture everything (albeit, slightly over-counting task CPU
time by including any enqueuing work done before we hit `wait()`).
The various metrics counters could be a separate inlined struct if we
think that's a useful abstraction, but for now i've just put them
directly in `jl_task_t`. They are all atomic, except the
`metrics_enabled` flag itself (which we now have to check on task
start/switch/done even if metrics are not enabled) which is set on task
construction and marked `const` on the julia side.
In future PRs we could add more per-task metrics, e.g. compilation time,
GC time, allocations, potentially a wait-time breakdown (time waiting on
locks, channels, in the scheduler run queue, etc.), potentially the
number of yields.
Perhaps in future there could be ways to enable this on a per-thread and
per-task basis. And potentially in future these same timings could be
used by `@time` (e.g. writing this same timing data to a ScopedValue
like in https://github.com/JuliaLang/julia/pull/55103 but only for tasks
lexically scoped to inside the `@time` block).
Timings are off by default but can be turned on globally via starting
Julia with `--task-metrics=yes` or calling
`Base.Experimental.task_metrics(true)`. Metrics are collected for all
tasks created when metrics are enabled. In other words,
enabling/disabling timings via `Base.Experimental.task_metrics` does not
affect existing `Task`s, only new `Task`s.
The other new APIs are `Base.Experimental.task_running_time_ns(::Task)`
and `Base.Experimental.task_wall_time_ns(::Task)` for retrieving the new
metrics. These are safe to call on any task (including the current task,
or a task running on another thread). All these are in
`Base.Experimental` to give us room to change up the APIs as we add more
metrics in future PRs (without worrying about release timelines).
cc @NHDaly @kpamnany @d-netto
---------
Co-authored-by: Pete Vilter <pete.vilter@gmail.com>
Co-authored-by: K Pamnany <kpamnany@users.noreply.github.com>
Co-authored-by: Nathan Daly <nathan.daly@relational.ai>
Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
* Fully outline all GlobalRefs (#56746)
This is an alternative to #56714 that goes in the opposite direction -
just outline all GlobalRefs during lowering. It is a lot simpler that
#56714 at the cost of some size increase. Numbers:
sys.so .ldata size:
This PR: 159.8 MB
Master: 158.9 MB
I don't have numbers of #56714, because it's not fully complete.
Additionally, it's possible that restricting GlobalRefs from arguments
position would let us use a more efficient encoding in the future.
* Add #54800 to NEWS (#56774)
Show glyphs for latex or emoji shortcodes being suggested in the REPL
* Add the actual datatype to the heapsnapshot. This groups objects of the same type together (#56596)
* Fix typos in docstring, comments, and news (#56778)
* Add sort for NTuples (#54494)
This is partially a reland of #46104, but without the controversial `sort(x) = sort!(copymutable(x))` and with some extensibility improvements. Implements #54489.
* precompile: don't waste memory on useless inferred code (#56749)
We never have a reason to reference this data again since we already
have native code generated for it, so it is simply wasting memory and
download space.
$ du -sh {old,new}/usr/share/julia/compiled
256M old
227M new
* various globals cleanups (#56750)
While doing some work on analyzing what code runs at toplevel, I found a
few things to simplify or fix:
- simplify float.jl loop not to call functions just defined to get back
the value just stored there
- add method to the correct checkbounds function (instead of a local)
- missing world push/pop around jl_interpret_toplevel_thunk after #56509
- jl_lineno use maybe missed in #53799
- add some debugging dump for scm_to_julia_ mistakes
* Fix test report alignment (#56789)
* 🤖 [master] Bump the Pkg stdlib from d84a1a38b to e7c37f342 (#56786)
* Bump JuliaSyntax to 0.4.10 (#56110)
* libgit2: update enums from v1.8.0 (#56764)
* fix `exp(weirdNaN)` (#56784)
Fixes https://github.com/JuliaLang/julia/issues/56782
Fix `exp(reinterpret(Float64, 0x7ffbb14880000000))` returning non-NaN
value. This should have minimal performance impact since it's already in
the fallback branch.
* Add note to Vararg/UnionAll warning about making it an error (#56662)
This warning message
```
WARNING: Wrapping `Vararg` directly in UnionAll is deprecated (wrap the tuple instead).
You may need to write `f(x::Vararg{T})` rather than `f(x::Vararg{<:T})` or `f(x::Vararg{T}) where T` instead of `f(x::Vararg{T} where T)`.
```
(last extended in #49558) seems clear enough if you wrote the code. But
if it's coming from 10 packages deep, there's no information to track
down the origin.
Turns out you can do this with `--depwarn=error`. But the message
doesn't tell you that, and doesn't call itself a deprecation warning at
all.
* Add a note clearifying option parsing (#56285)
To help with #56274
* Fix undefined symbol error in version script (#55363)
lld 17 and above by default error if symbols listed in the version
script are undefined. Julia has a few of these, as some symbols are
defined conditionally in Julia (e.g. based on OS), others perhaps have
been removed from Julia and other seem to be defined in other libraries.
Further the version script is used in linking three times, each time
linking together different objects and so having different symbols
defined.
Adding `-Wl,--undefined-version` is not a great solution as passing that
to ld < 2.40 errors and there doesn't seem to be a great way to check if
a linker supports this flag.
I don't know how to get around these errors for symbols like
`_IO_stdin_used` which Julia doesn't define and it seems to matter
whether or not they are exported, see
https://libc-alpha.sourceware.narkive.com/SevIQmU3/io-stdin-used-stripped-by-version-scripts.
So I've converted all undefined symbols into wildcards to work around
the error.
Fixes #50414, fixes #54533 and replaces #55319.
---------
Co-authored-by: Zentrik <Zentrik@users.noreply.github.com>
* Update repr docstring to hint what it stands for. (#56761)
- Change `repr` docstring from "Create a string from any
value ..." to "Create a string representation of any value ...".
- Document that it typically emits parseable Julia code
---------
Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
* Fix generate_precompile statement grouping & avoid defining new func (#56317)
* remove LinearAlgebra specific bitarray tests since they have moved to the external LinearAlgebra.jl repo (#56800)
Moved in https://github.com/JuliaLang/LinearAlgebra.jl/pull/1148.
* Fix partially_inline for unreachable (#56787)
* codegen: reduce recursion in cfunction generation (#56806)
The regular code-path for this was only missing the age_ok handling, so
add in that handling so we can delete the custom path here for the test
and some of the brokenness that implied.
* gc: simplify sweep_weak_refs logic (#56816)
[NFCI]
* gc: improve mallocarrays locality (#56801)
* Fix codegen not handling invoke exprs with Codeinstances iwith jl_fptr_sparam_addr invoke types. (#56817)
fixes https://github.com/JuliaLang/julia/issues/56739
I didn't succeed in making a test for this. The sole trigger seems to be
```julia
using HMMGradients
T = Float32
A = T[0.0 1.0 0.0; 0.0 0.5 0.5; 1.0 0.0 0.0]
t2tr = Dict{Int,Vector{Int}}[Dict(1 => [2]),]
t2IJ= HMMGradients.t2tr2t2IJ(t2tr)
Nt = length(t2tr)+1
Ns = size(A,1)
y = rand(T,Nt,Ns)
c = rand(Float32, Nt)
beta = backward(Nt,A,c,t2IJ,y)
gamma = posterior(Nt,t2IJ,A,y)
```
in @oscardssmith memorynew PR
One other option is to have the builtin handle receiving a CI. That
might make the code cleaner and does handle the case where we receive a
dynamic CI (is that even a thing)
* 🤖 [master] Bump the SparseArrays stdlib from 1b4933c to 4fd3aad (#56830)
Stdlib: SparseArrays
URL: https://github.com/JuliaSparse/SparseArrays.jl.git
Stdlib branch: main
Julia branch: master
Old commit: 1b4933c
New commit: 4fd3aad
Julia version: 1.12.0-DEV
SparseArrays version: 1.12.0
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaSparse/SparseArrays.jl/compare/1b4933ccc7b1f97427ff88bd7ba58950021f2c60...4fd3aad5735e3b80eefe7b068f3407d7dd0c0924
```
$ git log --oneline 1b4933c..4fd3aad
4fd3aad Generalize `istriu`/`istril` to accept a band index (#590)
780c4de Bump codecov/codecov-action from 4 to 5 (#589)
1beb0e4 Update LICENSE.md (#587)
268d390 QR: handle xtype/dtype returned from LibSuiteSparse that don't match matrix element type (#586)
9731aef get rid of UUID changing stuff (#582)
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* Missing tests for IdSet (#56258)
Co-authored-by: CY Han <git@wo-class.cn>
* Fix eltype of flatten of tuple with non-2 length (#56802)
In 4c076c80af, eltype of flatten of tuple was improved by computing a
refined eltype at compile time. However, this implementation only worked
for length-2 tuples, and errored for all others.
Generalize this to all tuples.
Closes #56783
---------
Co-authored-by: Neven Sajko <s@purelymail.com>
* Skip or loosen two `errorshow` tests on 32-bit Windows (#56837)
Ref #55900.
* Adding tests for AbstractArrayMath.jl (#56773)
added tests for lines 7, 137-154 (insertdims function) from
base/abstractarraymath.jl
---------
Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com>
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* 🤖 [master] Bump the Downloads stdlib from 89d3c7d to afd04be (#56826)
Stdlib: Downloads
URL: https://github.com/JuliaLang/Downloads.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 89d3c7d
New commit: afd04be
Julia version: 1.12.0-DEV
Downloads version: 1.6.0(It's okay that it doesn't match)
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/Downloads.jl/compare/89d3c7dded535a77551e763a437a6d31e4d9bf84...afd04be8aa94204c075c8aec83fca040ebb4ff98
```
$ git log --oneline 89d3c7d..afd04be
afd04be Bump codecov/codecov-action from 4 to 5 (#264)
39036e1 CI: Use Dependabot to automatically update external GitHub Actions (#263)
78e7c7c Bump CI actions versions (#252)
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* 🤖 [master] Bump the SHA stdlib from aaf2df6 to 8fa221d (#56834)
Stdlib: SHA
URL: https://github.com/JuliaCrypto/SHA.jl.git
Stdlib branch: master
Julia branch: master
Old commit: aaf2df6
New commit: 8fa221d
Julia version: 1.12.0-DEV
SHA version: 0.7.0(Does not match)
Bump invoked by: @inkydragon
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaCrypto/SHA.jl/compare/aaf2df61ff8c3898196587a375d3cf213bd40b41...8fa221ddc8f3b418d9929084f1644f4c32c9a27e
```
$ git log --oneline aaf2df6..8fa221d
8fa221d ci: update doctest config (#120)
346b359 ci: Update ci config (#115)
aba9014 Fix type mismatch for `shake/digest!` and setup x86 ci (#117)
0b76d04 Merge pull request #114 from JuliaCrypto/dependabot/github_actions/codecov/codecov-action-5
5094d9d Update .github/workflows/CI.yml
45596b1 Bump codecov/codecov-action from 4 to 5
230ab51 test: remove outdate tests (#113)
7f25aa8 rm: Duplicated const alias (#111)
aa72f73 [SHA3] Fix padding special-case (#108)
3a01401 Delete Manifest.toml (#109)
da351bb Remvoe all getproperty funcs (#99)
4eee84f Bump codecov/codecov-action from 3 to 4 (#104)
15f7dbc Bump codecov/codecov-action from 1 to 3 (#102)
860e6b9 Bump actions/checkout from 2 to 4 (#103)
8e5f0ea Add dependabot to auto update github actions (#100)
4ab324c Merge pull request #98 from fork4jl/sha512-t
a658829 SHA-512: add ref to NIST standard
11a4c73 Apply suggestions from code review
969f867 Merge pull request #97 from fingolfin/mh/Vector
b1401fb SHA-512: add NIST test
4d7091b SHA-512: add to docs
09fef9a SHA-512: test SHA-512/224, SHA-512/256
7201b74 SHA-512: impl SHA-512/224, SHA-512/256
4ab85ad Array -> Vector
8ef91b6 fixed bug in padding for shake, addes testcases for full code coverage (#95)
88e1c83 Remove non-existent property (#75)
068f85d shake128,shake256: fixed typo in export declarations (#93)
176baaa SHA3 xof shake128 and shake256 (#92)
e1af7dd Hardcode doc edit backlink
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* 🤖 [master] Bump the StyledStrings stdlib from 056e843 to 8985a37 (#56832)
Stdlib: StyledStrings
URL: https://github.com/JuliaLang/StyledStrings.jl.git
Stdlib branch: main
Julia branch: master
Old commit: 056e843
New commit: 8985a37
Julia version: 1.12.0-DEV
StyledStrings version: 1.11.0(Does not match)
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/StyledStrings.jl/compare/056e843b2d428bb9735b03af0cff97e738ac7e14...8985a37ac054c37d084a03ad2837208244824877
```
$ git log --oneline 056e843..8985a37
8985a37 Fix interpolation edge case dropping annotations
729f56c Add typeasserts to `convert(::Type{Face}, ::Dict)`
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* xref `UnionAll` in the doc string of `where` (#56411)
* 🤖 [master] Bump the ArgTools stdlib from 997089b to 1314758 (#56839)
Stdlib: ArgTools
URL: https://github.com/JuliaIO/ArgTools.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 997089b
New commit: 1314758
Julia version: 1.12.0-DEV
ArgTools version: 1.1.2(Does not match)
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaIO/ArgTools.jl/compare/997089b9cd56404b40ff766759662e16dc1aab4b...1314758ad02ff5e9e5ca718920c6c633b467a84a
```
$ git log --oneline 997089b..1314758
1314758 build(deps): bump codecov/codecov-action from 4 to 5; also, use CODECOV_TOKEN (#40)
68bb888 Fix typo in `arg_write` docstring (#39)
5d56027 build(deps): bump julia-actions/setup-julia from 1 to 2 (#38)
b6189c7 build(deps): bump codecov/codecov-action from 3 to 4 (#37)
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* 🤖 [master] Bump the Statistics stdlib from 68869af to d49c2bf (#56831)
Stdlib: Statistics
URL: https://github.com/JuliaStats/Statistics.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 68869af
New commit: d49c2bf
Julia version: 1.12.0-DEV
Statistics version: 1.11.2(Does not match)
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaStats/Statistics.jl/compare/68869af06e8cdeb7aba1d5259de602da7328057f...d49c2bf4f81e1efb4980a35fe39c815ef8396297
```
$ git log --oneline 68869af..d49c2bf
d49c2bf Merge pull request #178 from JuliaStats/dw/ci
d10d6a3 Update Project.toml
1b67c17 Merge pull request #168 from JuliaStats/andreasnoack-patch-2
c3721ed Add a coverage badge
8086523 Test earliest supported Julia version and prereleases
12a1976 Update codecov in ci.yml
2caf0eb Merge pull request #177 from JuliaStats/ViralBShah-patch-1
33e6e8b Update ci.yml to use julia-actions/cache
a399c19 Merge pull request #176 from JuliaStats/dependabot/github_actions/julia-actions/setup-julia-2
6b8d58a Merge branch 'master' into dependabot/github_actions/julia-actions/setup-julia-2
c2fb201 Merge pull request #175 from JuliaStats/dependabot/github_actions/actions/cache-4
8f808e4 Merge pull request #174 from JuliaStats/dependabot/github_actions/codecov/codecov-action-4
7f82133 Merge pull request #173 from JuliaStats/dependabot/github_actions/actions/checkout-4
046fb6f Update ci.yml
c0fc336 Bump julia-actions/setup-julia from 1 to 2
a95a57a Bump actions/cache from 1 to 4
b675501 Bump codecov/codecov-action from 1 to 4
0088c49 Bump actions/checkout from 2 to 4
ad95c08 Create dependabot.yml
40275e2 Merge pull request #167 from JuliaStats/andreasnoack-patch-1
fa5592a Merge pull request #170 from mbauman/patch-1
cf57562 Add more tests of mean and median of ranges
128dc11 Merge pull request #169 from stevengj/patch-1
48d7a02 docfix: abs2, not ^2
2ac5bec correct std docs: sqrt is elementwise
39f6332 Merge pull request #96 from josemanuel22/mean_may_return_incorrect_results
db3682b Merge branch 'master' into mean_may_return_incorrect_results
9e96507 Update src/Statistics.jl
58e5986 Test prereleases
6e76739 Implement one-argument cov2cor!
b8fee00 Stop testing on nightly
9addbb8 Merge pull request #162 from caleb-allen/patch-1
6e3d223 Merge pull request #164 from aplavin/patch-1
71ebe28 Merge pull request #166 from JuliaStats/dw/cov_cor_optimization
517afa6 add tests
aa0f549 Optimize `cov` and `cor` with identical arguments
cc11ea9 propagate NaN value in median
cf7040f Use non-mobile Wikipedia urls
547bf4d adding docu to mean! explain target should not alias with the source
296650a adding docu to mean! explain target should not alias with the source
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* inference: don't allocate `TryCatchFrame` for `compute_trycatch(::IRCode)` (#56835)
`TryCatchFrame` is only required for the abstract interpretation and is
not necessary in `compute_trycatch` within slot2ssa.jl.
@nanosoldier `runbenchmarks("inference", vs=":master")`
* 🤖 [master] Bump the LazyArtifacts stdlib from e9a3633 to a719c0e (#56827)
Stdlib: LazyArtifacts
URL: https://github.com/JuliaPackaging/LazyArtifacts.jl.git
Stdlib branch: main
Julia branch: master
Old commit: e9a3633
New commit: a719c0e
Julia version: 1.12.0-DEV
LazyArtifacts version: 1.11.0(Does not match)
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaPackaging/LazyArtifacts.jl/compare/e9a36338d5d0dfa4b222f4e11b446cbb7ea5836c...a719c0e3d68a95c6f3dc9571459428ca8761fa2c
```
$ git log --oneline e9a3633..a719c0e
a719c0e Add compat for LazyArtifacts
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* make `memorynew` intrinsic (#56803)
Attempt to split up https://github.com/JuliaLang/julia/pull/55913 into 2
pieces. This piece now only adds the `memorynew` intrinsic without any
of the optimizations enabled by
https://github.com/JuliaLang/julia/pull/55913. As such, this PR should
be ready to merge now. (and will make
https://github.com/JuliaLang/julia/pull/55913 smaller and simpler)
---------
Co-authored-by: gbaraldi <baraldigabriel@gmail.com>
* fix `compute_trycatch` for `IRCode` /w `EnterNode` with `catch_dest==0` (#56846)
* EA: general cleanup (#56848)
The handling of `Array` in EA was implemented before `Memory` was
introduced and has now become stale. Analysis for `Array` should be
reintroduced after the upcoming overhaul is completed. For now, I will
remove the existing stale code.
* 🤖 [master] Bump the Distributed stdlib from 6c7cdb5 to c613685 (#56825)
Stdlib: Distributed
URL: https://github.com/JuliaLang/Distributed.jl
Stdlib branch: master
Julia branch: master
Old commit: 6c7cdb5
New commit: c613685
Julia version: 1.12.0-DEV
Distributed version: 1.11.0(Does not match)
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/Distributed.jl/compare/6c7cdb5860fa5cb9ca191ce9c52a3d25a9ab3781...c6136853451677f1957bec20ecce13419cde3a12
```
$ git log --oneline 6c7cdb5..c613685
c613685 Merge pull request #116 from JuliaLang/ci-caching
20e2ce7 Use julia-actions/cache in CI
9c5d73a Merge pull request #112 from JuliaLang/dependabot/github_actions/codecov/codecov-action-5
ed12496 Merge pull request #107 from JamesWrigley/remotechannel-empty
010828a Update .github/workflows/ci.yml
11451a8 Bump codecov/codecov-action from 4 to 5
8b5983b Merge branch 'master' into remotechannel-empty
729ba6a Fix docstring of `@everywhere` (#110)
af89e6c Adding better docs to exeflags kwarg (#108)
8537424 Implement Base.isempty(::RemoteChannel)
6a0383b Add a wait(::[Abstract]WorkerPool) (#106)
1cd2677 Bump codecov/codecov-action from 1 to 4 (#96)
cde4078 Bump actions/cache from 1 to 4 (#98)
6c8245a Bump julia-actions/setup-julia from 1 to 2 (#97)
1ffaac8 Bump actions/checkout from 2 to 4 (#99)
8e3f849 Fix RemoteChannel iterator interface (#100)
f4aaf1b Fix markdown errors in README.md (#95)
2017da9 Merge pull request #103 from JuliaLang/sf/sigquit_instead
07389dd Use `SIGQUIT` instead of `SIGTERM`
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* inference: handle cases where `:the_exception` is used independently (#56836)
`Expr(:the_exception)` is handled by the interpreter in all cases,
however, inference assumes it is only used within frames containing
`try/catch`. This commit corrects that assumption.
* precompileplkgs: release parallel limiter when waiting for another process (#56844)
* Tests for two AbstractString methods (#56247)
Co-authored-by: Chengyu Han <cyhan.dev@outlook.com>
* Tests for annotated string/char c-tors (#56513)
Co-authored-by: Chengyu Han <cyhan.dev@outlook.com>
* docs: fix example for `sort` and add new doctest (#56843)
* jitlayers: re-delete deregisterEHFrames impl (#56854)
Fixes https://github.com/maleadt/LLVM.jl/issues/496
* docs: fix edge case in rational number conversion `float(a//b)` (#56772)
Fixes #56726 added the changes that were suggested. fixing the mistake.
---------
Co-authored-by: Max Horn <max@quendi.de>
Co-authored-by: Chengyu Han <git@wo-class.cn>
* docs: fix scope type of a `struct` to hard (#56755)
Is struct not a hard scope?
```jl
julia> b = 1
struct Tester
tester
Tester(tester) = new(tester)
b = 2
Tester() = new(b)
end
b
1
```
* dict docs: document that ordering of keys/values/pairs match iterate (#56842)
Fix #56841.
Currently the documentation states that keys(dict) and values(dict)
iterate in the same order. But it is not stated whether this is the same
order as that used by pairs(dict), or when looping, for (k,v) in dict.
This PR makes this guarantee explicit.
* fix abstract_eval_cfunction mistakes (#56856)
Noticed in code reading, that 35e4a1f9689f4b98f301884e0683e4f07db7514b
simplified this incorrectly resulting in all arguments being assigned to
the function type, and then 7f8635f11cae5f3f592afcc7b55c8e0e23589c3d
further broke the return type expected for the failure case.
Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com>
* inference: use `ssa_def_slot` for `typeassert` refinement (#56859)
Allows type refinement in the following kind of case:
```julia
julia> @test Base.infer_return_type((Vector{Any},)) do args
codeinst = first(args)
if codeinst isa Core.MethodInstance
mi = codeinst
else
codeinst::Core.CodeInstance
mi = codeinst.def
end
return mi
end == Core.MethodInstance
Test Passed
```
* EA: use embedded `CodeInstance` directly for escape cache lookup (#56860)
* Compiler.jl: use `Base.[all|any]` instead of `Compiler`'s own versions (#56851)
The current `Compiler` defines its own versions of `all` and `any`,
which are separate generic functions from `Base.[all|any]`:
https://github.com/JuliaLang/julia/blob/2ed1a411e0a080f3107e75bb65105a15a0533a90/Compiler/src/utilities.jl#L15-L32
On the other hand, at the point where `Base.Compiler` is bootstrapped,
only a subset of `Base.[all|any]` are defined, specifically those
related to `Tuple`:
https://github.com/JuliaLang/julia/blob/2ed1a411e0a080f3107e75bb65105a15a0533a90/base/tuple.jl#L657-L668.
Consequently, in the type inference world, functions like
`Base.all(::Generator)` are unavailable. If `Base.Compiler` attempts to
perform operations such as `::BitSet ⊆ ::BitSet` (which internally uses
`Base.[all|any]`), a world age error occurs (while `Compiler.[all|any]`
can handle these operations, `::BitSet ⊆ ::BitSet` uses
`Base.[all|any]`, leading to this issue)
To resolve this problem, this commit removes the custom `Compiler`
versions of `[all|any]` and switches to using the Base versions.
One concern is that the previous `Compiler` versions of `[all|any]`
utilized `@nospecialize`. That annotation was introduced a long time ago
to prevent over-specialization, but it is questionable whether it is
still effective with the current compiler implementation. The results of
the nanosoldier benchmarks conducted below also seem to confirm that
the `@nospecialize`s are no longer necessary for those functions.
* Extend `Base.rationalize` instead of defining new function (#56793)
#55886 accidentally created a new function
`Base.MathConstants.rationalize` instead of extending
`Base.rationalize`, which is the reason why `Base.rationalize(Int, π)`
isn’t constant-folded in Julia 1.10 and 1.11:
```
julia> @btime rationalize(Int,π);
1.837 ns (0 allocations: 0 bytes) # v1.9: constant-folded
88.416 μs (412 allocations: 15.00 KiB) # v1.10: not constant-folded
```
This PR fixes that. It should probably be backported to 1.10 and 1.11.
* 🤖 [master] Bump the LinearAlgebra stdlib from 56d561c to 1137b4c (#56828)
Stdlib: LinearAlgebra
URL: https://github.com/JuliaLang/LinearAlgebra.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 56d561c
New commit: 1137b4c
Julia version: 1.12.0-DEV
LinearAlgebra version: 1.11.0(Does not match)
Bump invoked by: @DilumAluthge
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/LinearAlgebra.jl/compare/56d561c22e1ab8e0421160edbdd42f3f194ecfa8...1137b4c7fa8297cef17c4ae0982d7d89d4ab7dd8
```
$ git log --oneline 56d561c..1137b4c
1137b4c Port structured opnorm changes from main julia PR (#1138)
ade3654 port bitarray tests to LinearAlgebra.jl (#1148)
130b94a Fix documentation bug in QR docstring (#1145)
128518e setup coverage (#1144)
15f7e32 ci: add linux-i686 (#1142)
b6f87af Fallback `newindex` method with a `BandIndex` (#1143)
d1e267f bring back [l/r]mul! shortcuts for (kn)own triangular types (#1137)
5cdeb46 don't refer to internal variable names in gemv exceptions (#1141)
4a3dbf8 un-revert "Simplify some views of Adjoint matrices" (#1122)
7b34d81 [CI] Install DependaBot (#1115)
f567112 Port structured opnorm changes from main julia PR Originally written by mcognetta
d406524 implements a `rank(::SVD)` method and adds unit tests. fixes #1126 (#1127)
f13f940 faster implementation of rank(::QRPivoted) fixes #1128 (#1129)
8ab7e09 Merge pull request #1132 from JuliaLang/jishnub/qr_ldiv_R_cache
85919e6 Merge pull request #1108 from JuliaLang/jishnub/tri_muldiv_stride
195e678 Cache and reuse `R` in adjoint `QR` `ldiv!`
cd0da66 Update comments LAPACK -> BLAS
b3ec55f Fix argument name in stride
d3a9a3e Non-contiguous matrices in triangular mul and div
aecb714 Reduce number of test combinations in test/triangular.jl (#1123)
62e45d1 Update .github/dependabot.yml
ff78c38 use an explicit file extension when creating sysimage (#1119)
43b541e try use updated windows compilers in PackageCompiler (#1120)
c00cb77 [CI] Install DependaBot
b285b1c Update LICENSE (#1113)
7efc3ba remove REPL from custom sysimage (#1112)
b7f82ec add a README
```
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* Add ABIOverride type for def field (#56555)
Together with #54899, this PR is intending to replicate the
functionality of #54373, which allowed particular specializations to
have a different ABI signature than what would be suggested by the
MethodInstance's `specTypes` field. This PR handles that by adding a
special `ABIOverwrite` type, which, when placed in the `owner` field of
a `CodeInstance` instructs the system to use the given signature
instead.
* EAUtils: perform `code_escapes` with a global cache by default (#56868)
In JuliaLang/julia#56860, `EAUtils.EscapeAnalyzer` was updated to create
a new cache for each invocation of `code_escapes`, similar to how
Cthulhu.jl behaves. However, `code_escapes` is often used for
performance analysis like `@benchmark code_escapes(...)`, and in such
cases, a large number of `CodeInstance`s can be generated for each
invocation. This could potentially impact native code execution.
So this commit changes the default behavior so that `code_escapes` uses
the same pre-existing cache by default. We can still opt-in to perform a
fresh analysis by specifying
`cache_token=EAUtils.EscapeAnalyzerCacheToken()`.
* Small ABIOverride follow up and add basic test (#56877)
Just a basic `invoke` test for now. There's various other code paths
that should also be tested with ABI overwrites, but this gives us the
basic framework and more tests can be added as needed.
* Package docstring: more peaceful README introduction (#56798)
Hello! 👋
#39093 is great! This PR styles the banner ("Displaying contents of
readme found at ...") differently:
Before:
```
help?> DifferentialEquations
search: DifferentialEquations
No docstring found for module DifferentialEquations.
Exported names
≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡
@derivatives, @ode_def, @ode_def_all, @ode_def_bare,
... (manually truncated)
Displaying contents of readme found at /Users/ian/.julia/packages/DifferentialEquations/HSWeG/README.md
≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡
DifferentialEquations.jl
≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡
(Image: Join the chat at https://gitter.im/JuliaDiffEq/Lobby) (https://gitter.im/JuliaDiffEq/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) (Image:
Build Status) (https://github.com/SciML/DifferentialEquations.jl/actions?query=workflow%3ACI) (Image: Stable) (http://diffeq.sciml.ai/stable/) (Image: Dev) (http://diffeq.sciml.ai/dev/)
(Image: DOI) (https://zenodo.org/badge/latestdoi/58516043)
This is a suite for numerically solving differential equations written in Julia and available for use in Julia, Python, and R. The purpose of this package is to supply efficient Julia
implementations of solvers for various differential equations. Equations within the realm of this package include:
• Discrete equations (function maps, discrete stochastic (Gillespie/Markov) simulations)
• Ordinary differential equations (ODEs)
```
After:
```
help?> DifferentialEquations
search: DifferentialEquations
No docstring found for module DifferentialEquations.
Exported names
≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡
@derivatives, @ode_def, @ode_def_all, @ode_def_bare,
... (manually truncated)
────────────────────────────────────────────────────────────────────────────
Package description from README.md:
DifferentialEquations.jl
≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡
(Image: Join the chat at https://gitter.im/JuliaDiffEq/Lobby) (https://gitter.im/JuliaDiffEq/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) (Image:
Build Status) (https://github.com/SciML/DifferentialEquations.jl/actions?query=workflow%3ACI) (Image: Stable) (http://diffeq.sciml.ai/stable/) (Image: Dev) (http://diffeq.sciml.ai/dev/)
(Image: DOI) (https://zenodo.org/badge/latestdoi/58516043)
This is a suite for numerically solving differential equations written in Julia and available for use in Julia, Python, and R. The purpose of this package is to supply efficient Julia
implementations of solvers for various differential equations. Equations within the realm of this package include:
• Discrete equations (function maps, discrete stochastic (Gillespie/Markov) simulations)
• Ordinary differential equations (ODEs)
```
I think this makes it a bit less technical (the word "Displaying" and
the long filename), and it improves the reading flow.
* Support indexing `Broadcasted` objects using `Integer`s (#56470)
This adds support for `IndexLinear` `eachindex`, as well as
bounds-checking and `getindex` for a `Broadcasted` with an `Integer`
index.
Instead of using the number of dimensions in `eachindex` to determine
whether to use `CartesianIndices`, we may use the `IndexStyle`. This
should not change anything currently, but this adds the possibility of
`nD` broadcasting using linear indexing if the `IndexStyle` of the
`Broadcasted` object is `IndexLinear`.
After this,
```julia
julia> bc = Broadcast.broadcasted(+, reshape(1:4, 2, 2), 1:2)
Base.Broadcast.Broadcasted(+, ([1 3; 2 4], 1:2))
julia> eachindex(bc)
CartesianIndices((2, 2))
julia> eachindex(IndexLinear(), bc)
Base.OneTo(4)
julia> [bc[I] for I in eachindex(IndexLinear(), bc)]
4-element Vector{Int64}:
2
4
4
6
julia> vec(collect(bc))
4-element Vector{Int64}:
2
4
4
6
```
This PR doesn't add true linear indexing support for `IndexCartesian`
`Broadcasted` objects. In such cases, an `Integer` index is converted to
a `CartesianIndex` before it is used in indexing.
* Limit the scope of `@inbounds` in `searchsorted*` (#56882)
This removes bounds-checking only in the indexing operation, instead of
for the entire block.
* Remove trailing comma in 0-dim `reshape` summary (#56853)
Currently, there is an extra comma in displaying the summary for a 0-dim
`ReshapedArray`:
```julia
julia> reshape(1:1)
0-dimensional reshape(::UnitRange{Int64}, ) with eltype Int64:
1
```
This PR only prints the comma if `dims` isn't empty, so that we now
obtain
```julia
julia> reshape(1:1)
0-dimensional reshape(::UnitRange{Int64}) with eltype Int64:
1
```
* Switch debug registry mapping to `CodeInstance` (#56878)
Currently our debugging code maps from pointers to `MethodInstance`.
However, I think it makes more sense to map to `CodeInstance` instead,
since that's what we're actually compiling and as we're starting to make
more use of external owners and custom specializations, we'll want to be
able to annotate that in backtraces. This only adjusts the internal data
structures - any actual printing changes for those sorts of use cases
will have to come separately.
* binaryplatforms.jl: Remove duplicated riscv64 entry (#56888)
* Minor documentation updates (#56883)
- Array/Vector/Matrix output showing
- Syntax highlighting for fenced code block examples
---------
Co-authored-by: Chengyu Han <cyhan.dev@outlook.com>
Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com>
* Remove `similar` specialization with `Integer`s (#56881)
This method seems unnecessary, as the method on line 830 does the same
thing. Having this method in `Base` requires packages to disambiguate
against this by adding redundant methods.
* simplify and slightly improve memorynew inference (#56857)
while investigating some missed optimizations in
https://github.com/JuliaLang/julia/pull/56847, @gbaraldi and I realized
that `copy(::Array)` was using `jl_genericmemory_copy_slice` rather than
the `memmove`/`jl_genericmemory_copyto` that `copyto!` lowers to. This
version lets us use the faster LLVM based Memory initialization, and the
memove can theoretically be further optimized by LLVM (e.g. not copying
elements that get over-written without ever being read).
```
julia> @btime copy($[1,2,3])
15.521 ns (2 allocations: 80 bytes) # before
12.116 ns (2 allocations: 80 bytes) #after
julia> m = Memory{Int}(undef, 3);
julia> m.=[1,2,3];
julia> @btime copy($m)
11.013 ns (1 allocation: 48 bytes) #before
9.042 ns (1 allocation: 48 bytes) #after
```
We also optimize the `memorynew` type inference to make it so that
getting the length of a memory with known length will propagate that
length information (which is important for cases like `similar`/`copy`
etc).
* Update the OffsetArrays test helper (#56892)
This updates the test helper to match v1.15.0 of `OffsetArrays.jl`. The
current version was copied over from v1.11.2, which was released on May
20, 2022, so this bump fetches the intermediate updates to the package.
The main changes are updates to `unsafe_wrap` and `reshape`. In the
latter, several redundant methods are now removed, whereas, in the
former, methods are added to wrap a `Ptr` in an `OffsetArray`
(https://github.com/JuliaArrays/OffsetArrays.jl/issues/275#issue-1222585268).
A third major change is that an `OffsetArray` now shares its parent's
`eltype` and `ndims` in the struct definition, whereas previously this
was ensured through the constructor. Other miscellaneous changes are
included, such as performance-related ones.
* fix: Base.GMP.MPZ.invert yielding return code instead of return value (#56894)
There is a bug in `Base.GMP.MPZ.invert` it returned GMP return code,
instead of the actual value. This commit fixes it.
Before:
```
julia> Base.GMP.MPZ.invert(big"3", big"7")
1
```
After:
```
julia> Base.GMP.MPZ.invert(big"3", big"7")
5
```
* fix precompilation error printing if `CI` is set (#56905)
Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
* Add #53664, public&export error to news (#56911)
Suggested by @simeonschaub
[here](https://github.com/JuliaLang/julia/pull/53664#issuecomment-2562408676)
* restrict `optimize_until` argument type correctly (#56912)
The object types that `optimize_until` can accept are restricted by
`matchpass`, so this restriction should also be reflected in functions
like `typeinf_ircode` too.
* doc: manual: give the Performance Tips a table of contents (#56917)
The page is quite long, and now it also has a more intricate structure
than before, IMO it deserves a TOC.
* ReentrantLock: wakeup a single task on unlock and add a short spin (#56814)
I propose a change in the implementation of the `ReentrantLock` to
improve its overall throughput for short critical sections and fix the
quadratic wake-up behavior where each unlock schedules **all** waiting
tasks on the lock's wait queue.
This implementation follows the same principles of the `Mutex` in the
[parking_lot](https://github.com/Amanieu/parking_lot/tree/master) Rust
crate which is based on the Webkit
[WTF::ParkingLot](https://webkit.org/blog/6161/locking-in-webkit/)
class. Only the basic working principle is implemented here, further
improvements such as eventual fairness will be proposed separately.
The gist of the change is that we add one extra state to the lock,
essentially going from:
```
0x0 => The lock is not locked
0x1 => The lock is locked by exactly one task. No other task is waiting for it.
0x2 => The lock is locked and some other task tried to lock but failed (conflict)
```
To:
```
# PARKED_BIT | LOCKED_BIT | Description
# 0 | 0 | The lock is not locked, nor is anyone waiting for it.
# -----------+------------+------------------------------------------------------------------
# 0 | 1 | The lock is locked by exactly one task. No other task is
# | | waiting for it.
# -----------+------------+------------------------------------------------------------------
# 1 | 0 | The lock is not locked. One or more tasks are parked.
# -----------+------------+------------------------------------------------------------------
# 1 | 1 | The lock is locked by exactly one task. One or more tasks are
# | | parked waiting for the lock to become available.
# | | In this state, PARKED_BIT is only ever cleared when the cond_wait lock
# | | is held (i.e. on unlock). This ensures that
# | | we never end up in a situation where there are parked tasks but
# | | PARKED_BIT is not set (which would result in those tasks
# | | potentially never getting woken up).
```
In the current implementation we must schedule all tasks to cause a
conflict (state 0x2) because on unlock we only notify any task if the
lock is in the conflict state. This behavior means that with high
contention and a short critical section the tasks will be effectively
spinning in the scheduler queue.
With the extra state the proposed implementation has enough information
to know if there are other tasks to be notified or not, which means we
can always notify one task at a time while preserving the optimized path
of not notifying if there are no tasks waiting. To improve throughput
for short critical sections we also introduce a bounded amount of
spinning before attempting to park.
### Results
Not spinning on the scheduler queue greatly reduces the CPU utilization
of the following example:
```julia
function example()
lock = ReentrantLock()
@sync begin
for i in 1:10000
Threads.@spawn begin
@lock lock begin
sleep(0.001)
end
end
end
end
end
@time example()
```
Current:
```
28.890623 seconds (101.65 k allocations: 7.646 MiB, 0.25% compilation time)
```
Proposed:
```
22.806669 seconds (101.65 k allocations: 7.814 MiB, 0.35% compilation time)
```
In a micro-benchmark where 8 threads contend for a single lock with a
very short critical section we see a ~2x improvement.
Current:
```
8-element Vector{Int64}:
6258688
5373952
6651904
6389760
6586368
3899392
5177344
5505024
Total iterations: 45842432
```
Proposed:
```
8-element Vector{Int64}:
12320768
12976128
10354688
12845056
7503872
13598720
13860864
11993088
Total iterations: 95453184
```
~~In the uncontended scenario the extra bookkeeping causes a 10%
throughput reduction:~~
EDIT: I reverted _trylock to the simple case to recover the uncontended
throughput and now both implementations are on the same ballpark
(without hurting the above numbers).
In the uncontended scenario:
Current:
```
Total iterations: 236748800
```
Proposed:
```
Total iterations: 237699072
```
Closes #56182
* 🤖 [master] Bump the Pkg stdlib from e7c37f342 to c7e611bc8 (#56918)
* Don't report only-inferred methods as recompiles (#56914)
* precompilepkgs: respect loaded dependencies when precompiling for load (#56901)
* Add julia-repl in destructuring docs (#56866)
* doc: manual: Command-line Interface: tiny Markdown code formatting fix (#56919)
* Make threadpoolsize(), threadpooltids(), and ngcthreads() public (#55701)
* [Test] Print RNG of a failed testset and add option to set it (#56260)
Also, add a keyword option to `@testset` to let users override the seed
used there, to make testsets more replicable.
To give you a taster of what this PR
enables:
```
julia> using Random, Test
julia> @testset begin
@test rand() == 0
end;
test set: Test Failed at REPL[2]:2
Expression: rand() == 0
Evaluated: 0.559472630416976 == 0
Stacktrace:
[1] top-level scope
@ REPL[2]:2
[2] macro expansion
@ ~/repo/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:1713 [inlined]
[3] macro expansion
@ REPL[2]:2 [inlined]
[4] macro expansion
@ ~/repo/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:679 [inlined]
Test Summary: | Fail Total Time
test set | 1 1 0.9s
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.
Random seed for this testset: Xoshiro(0x2e026445595ed28e, 0x07bb81ac4c54926d, 0x83d7d70843e8bad6, 0xdbef927d150af80b, 0xdbf91ddf2534f850)
julia> @testset rng=Xoshiro(0x2e026445595ed28e, 0x07bb81ac4c54926d, 0x83d7d70843e8bad6, 0xdbef927d150af80b, 0xdbf91ddf2534f850) begin
@test rand() == 0.559472630416976
end;
Test Summary: | Pass Total Time
test set | 1 1 0.0s
```
This also works with nested testsets, and testsets on for loops:
```
julia> @testset rng=Xoshiro(0xc380f460355639ee, 0xb39bc754b7d63bbf, 0x1551dbcfb5ed5668, 0x71ab5a18fec21a25, 0x649d0c1be1ca5436) "Outer" begin
@test rand() == 0.0004120194925605336
@testset rng=Xoshiro(0xee97f5b53f7cdc49, 0x480ac387b0527d3d, 0x614b416502a9e0f5, 0x5250cb36e4a4ceb1, 0xed6615c59e475fa0) "Inner: $(i)" for i in 1:10
@test rand() == 0.39321938407066637
end
end;
Test Summary: | Pass Total Time
Outer | 11 11 0.0s
```
Being able to see what was the seed inside a testset and being able to
set it afterwards should make replicating test failures which only
depend on the state of the RNG much easier to debug.
* 🤖 [master] Bump the Pkg stdlib from c7e611bc8 to 739a64a0b (#56927)
* 🤖 [master] Bump the Pkg stdlib from 739a64a0b to 8d3cf02e5 (#56930)
* doc: summary of output functions: cross-ref println and printstyled (#55860)
Slight tweak to #54547 to add cross-referenes to `println` and
`printstyled`.
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
* Make `Markdown.parse` public and add a docstring (#56818)
* precompilepkgs: color the "is currently loaded" text to make meaning clearer (#56926)
* Remove unused variable from `endswith` (#56934)
This is a trivial code cleanup suggestion. The `cub` variable was unused
in `endswith(::Union{String, SubString{String}}, ::Union{String,
SubString{String}})`.
I ran the tests with `make test`, there were some failures but they did
not appear to be related to the change.
* deps: support Unicode 16 via utf8proc 2.10.0 (#56925)
Similar to #51799, support [Unicode
16](https://www.unicode.org/versions/Unicode16.0.0/) by bumping utf8proc
to 2.10.0 (thanks to https://github.com/JuliaStrings/utf8proc/pull/277
by @eschnett).
This allows us to use [7 exciting new emoji
characters](https://www.unicode.org/emoji/charts-16.0/emoji-released.html)
as identifiers, including "face with bags under eyes"
`"\U1fae9"` (but still no superscript "q").
Closes #56035.
* teach jitlayers to use equivalent edges
Sometimes an edge (especially from precompile file, but sometimes from
inference) will specify a CodeInstance that does not need to be compiled
for its ABI and simply needs to be cloned to point to the existing copy
of it.
* opaque_closure: fix data-race mistakes with reading fields by using standard helper function
* opaque_closure: fix world-age mistake in fallback path
This was failing the h_world_age test sometimes.
* inference,codegen: connect source directly to jit
This avoids unnecessary compression when running (not generating code).
While generating code, we continue the legacy behavior of storing
compressed code, since restarting from a ji without that is quite slow.
Eventually, we should also remove that code also once we have generated
the object file from it.
This replaces the defective SOURCE_MODE_FORCE_SOURCE option with a new
`typeinf_ext_toplevel` batch-mode interface for compilation which
returns all required source code. Only two options remain now:
SOURCE_MODE_NOT_REQUIRED :
Require only that the IPO information (e.g. rettype and friends) is
present.
SOURCE_MODE_FORCE_ABI :
Require that the IPO information is present (for ABI computation)
and that the returned CodeInstance can be invoked on the host target
(preferably after inference, called directly, but perfectly
acceptable for Base.Compiler to instead force the runtime to use a
stub there or call into it with the interpreter instead by having
failed to provide any code).
This replaces the awkward `jl_create_native` interface (which is now
just a shim for calling the new batch-mode `typeinf_ext_toplevel`) with
a simpler `jl_emit_native` API, which does not do any inference or other
callbacks, but simply is a batch-mode call to `jl_emit_codeinfo` and
the work to build the external wrapper around them for linkage.
* delete unused code, so the jit no longer uses the inferred field at all
* Make sure we don't promise alignments that are larger than the heap alignment to LLVM (#56938)
Fixes https://github.com/JuliaLang/julia/issues/56937
---------
Co-authored-by: Oscar Smith <oscardssmith@gmail.com>
* 🤖 [master] Bump the NetworkOptions stdlib from 8eec5cb to c090626 (#56949)
Stdlib: NetworkOptions
URL: https://github.com/JuliaLang/NetworkOptions.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 8eec5cb
New commit: c090626
Julia version: 1.12.0-DEV
NetworkOptions version: 1.3.0(Does not match)
Bump invoked by: @giordano
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)
Diff:
https://github.com/JuliaLang/NetworkOptions.jl/compare/8eec5cb0acec4591e6db3c017f7499426cd8e352...c090626d3feee6d6a5c476346d22d6147c9c6d2d
```
$ git log --oneline 8eec5cb..c090626
c090626 Enable OpenSSL certificates (#36)
```
* [build] Fix value of `CMAKE_C{,XX}_ARG` when `CC_ARG` doesn't have args (#56920)
The problem was that `cut` by default prints the entire line if the
delimiter doesn't appear, unless the option `-s` is used, which means
that if `CC_ARG` contains only `gcc` then `CMAKE_CC_ARG` also contains
`gcc` instead of being empty.
Before the PR:
```console
$ make -C deps/ print-CC_ARG print-CMAKE_CC_ARG print-CMAKE_COMMON USECCACHE=1
make: Entering directory '/home/mose/repo/julia/deps'
CC_ARG=gcc
CMAKE_CC_ARG=gcc
CMAKE_COMMON=-DCMAKE_INSTALL_PREFIX:PATH=/home/mose/repo/julia/usr -DCMAKE_PREFIX_PATH=/home/mose/repo/julia/usr -DLIB_INSTALL_DIR=/home/mose/repo/julia/usr/lib -DCMAKE_INSTALL_LIBDIR=/home/mose/repo/julia/usr/lib -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER="$(which gcc)" -DCMAKE_C_COMPILER_ARG1="gcc " -DCMAKE_CXX_COMPILER="$(which g++)" -DCMAKE_CXX_COMPILER_ARG1="g++ " -DCMAKE_LINKER="$(which ld)" -DCMAKE_AR="$(which ar)" -DCMAKE_RANLIB="$(which ranlib)"
make: Leaving directory '/home/mose/repo/julia/deps'
```
After the PR
```console
$ make -C deps/ print-CC_ARG print-CMAKE_CC_ARG print-CMAKE_COMMON USECCACHE=1
make: Entering directory '/…
fingolfin
pushed a commit
to oscar-system/GenericCharacterTables.jl
that referenced
this pull request
Feb 14, 2025
This test currently relies on implicit world age increments at top level. We're re-evaluating where these go because julia is currently inconsistent about it in the interpreter, compiler and inference. To make sure this test keeps working on 1.12, add an explicit world age increment. See JuliaLang/julia#56509.
fingolfin
pushed a commit
to oscar-system/GenericCharacterTables.jl
that referenced
this pull request
Feb 14, 2025
This test currently relies on implicit world age increments at top level. We're re-evaluating where these go because julia is currently inconsistent about it in the interpreter, compiler and inference. To make sure this test keeps working on 1.12, add an explicit world age increment. See JuliaLang/julia#56509.
serenity4
pushed a commit
to serenity4/Diffractor.jl
that referenced
this pull request
Feb 21, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new, toplevel-only, syntax form
:worldincthat semantically represents the effect of raising the current task's world age to the latest world for the remainder of the current toplevel evaluation (that context being an entry toevalor a module expression). For detailed motivation on why this is desirable, see #55145, which I won't repeat here, but the gist is that we never really defined when world-age increments and worse are inconsistent about it. This is something we need to figure out now, because the bindings partition work will make world age even more observable via bindings.Having created a mechanism for world age increments, the big question is one of policy, i.e. when should these world age increments be inserted.
Several reasonable options exist:
callexpressionAs an example, case, consider
a == aat toplevel. Depending on the semantics that could either be the same as in local scope, or each of the four world age dependent lookups (three binding lookups, one method lookup) could (potentially) occur in a different world age.The general tradeoff here is between the risk of exposing the user to confusing world age errors and our ability to optimize top-level code (in general, any
:worldincstatement will require us to fully pessimize or recompile all following code).This PR basically implements option 2 with the following semantics:
:worldincexprs or after:moduleexprs.:worldincafter all struct definitions, method definitions,usingand `import.@evalmacro inserts a worldinc following the call toevalif at toplevelincludegains an implicitworldinc.Of these the fourth is probably the most questionable, but is necessary to make this non-breaking for most code patterns. Perhaps it would have been better to make
includea macro from the beginning (esp because it already has semantics that look a little like reaching into the calling module), but that ship has sailed.Unfortunately, I don't see any good intermediate options between this PR and option #3 above. I think option #3 is closest to what we have right now, but if we were to choose it and actually fix the soundness issues, I expect that we would be destroying all performance of global-scope code. For this reason, I would like to try to make the version in this PR work, even if the semantics are a little ugly.
The biggest pattern that this PR does not catch is:
We could apply the same
includespecial case to eval, but given the existence of@evalwhich allows addressing this at the macro level, I decided not to. We can decide which way we want to go on this based on what the package ecosystem looks like.