Segfault upon GC in function with try-catch block with type-unstable local variable #51852

topolarity · 2023-10-24T22:34:00Z

julia> versioninfo()
Julia Version 1.11.0-DEV.714
Commit 0be0b389e3* (2023-10-23 12:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: 12 × 12th Gen Intel(R) Core(TM) i7-1255U
  WORD_SIZE: 64
  LLVM: libLLVM-15.0.7 (ORCJIT, alderlake)
  Threads: 1 on 12 virtual cores
Environment:
  JULIA_PKG_SERVER = https://internal.juliahub.com
  JULIA_PKG_USE_CLI_GIT = true

function f(;kwargs...)
    try
        kwargs = rand((values(kwargs), (progress=1.0,)))
    catch
    end
    GC.gc()
    return kwargs
end

f(; progress="0.5") # seg-faults

GC error (probable corruption)
Allocations: 6031575 (Pool: 6024682; Big: 6893); GC: 10
#<intrinsic #25 slt_int>

thread 0 ptr queue:
~~~~~~~~~~ ptr queue top ~~~~~~~~~~
(Expr(:toplevel, #= Symbol("REPL[4]"):1 =#, Expr(:call, :include, "./mwe.jl")), 1)
==========
"include("./mwe.jl")"
==========
Expr(:toplevel, #= Symbol("REPL[4]"):1 =#, Expr(:call, :include, "./mwe.jl"))
==========
Base.GenericIOBuffer{Array{UInt8, 1}}(data=Array{UInt8, (0,)}[], reinit=true, readable=true, writable=true, seekable=true, append=false, size=0, maxsize=9223372036854775807, ptr=1, mark=-1)
==========
"0.5"
==========
~~~~~~~~~~ ptr queue bottom ~~~~~~~~~~

[26083] signal (6.-6): Aborted
in expression starting at /home/topolarity/repos/CedarEDA.jl/mwe.jl:11
pthread_kill at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
raise at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
gc_dump_queue_and_abort at /home/topolarity/repos/julia/src/gc.c:1933
gc_mark_outrefs at /home/topolarity/repos/julia/src/gc.c:2627 [inlined]
gc_mark_loop_serial_ at /home/topolarity/repos/julia/src/gc.c:2796
gc_mark_loop_serial at /home/topolarity/repos/julia/src/gc.c:2819
gc_mark_loop at /home/topolarity/repos/julia/src/gc.c:2932 [inlined]
_jl_gc_collect at /home/topolarity/repos/julia/src/gc.c:3263
ijl_gc_collect at /home/topolarity/repos/julia/src/gc.c:3569
gc at ./gcutils.jl:129 [inlined]
gc at ./gcutils.jl:129 [inlined]
#f#3 at /home/topolarity/repos/CedarEDA.jl/mwe.jl:7
f at /home/topolarity/repos/CedarEDA.jl/mwe.jl:2
unknown function (ip: 0x7ff254141b33)
_jl_invoke at /home/topolarity/repos/julia/src/gf.c:2911 [inlined]
ijl_apply_generic at /home/topolarity/repos/julia/src/gf.c:3093
jl_apply at /home/topolarity/repos/julia/src/julia.h:2002 [inlined]
do_call at /home/topolarity/repos/julia/src/interpreter.c:126
eval_value at /home/topolarity/repos/julia/src/interpreter.c:223
eval_stmt_value at /home/topolarity/repos/julia/src/interpreter.c:174 [inlined]
eval_body at /home/topolarity/repos/julia/src/interpreter.c:647
jl_interpret_toplevel_thunk at /home/topolarity/repos/julia/src/interpreter.c:787
jl_toplevel_eval_flex at /home/topolarity/repos/julia/src/toplevel.c:938
jl_toplevel_eval_flex at /home/topolarity/repos/julia/src/toplevel.c:881
ijl_toplevel_eval_in at /home/topolarity/repos/julia/src/toplevel.c:989
eval at ./boot.jl:398 [inlined]
include_string at ./loading.jl:2133
_jl_invoke at /home/topolarity/repos/julia/src/gf.c:2911 [inlined]
ijl_apply_generic at /home/topolarity/repos/julia/src/gf.c:3093
_include at ./loading.jl:2193
include at ./client.jl:497
unknown function (ip: 0x7ff25410a7c5)
_jl_invoke at /home/topolarity/repos/julia/src/gf.c:2911 [inlined]
ijl_apply_generic at /home/topolarity/repos/julia/src/gf.c:3093
jl_apply at /home/topolarity/repos/julia/src/julia.h:2002 [inlined]
do_call at /home/topolarity/repos/julia/src/interpreter.c:126
eval_value at /home/topolarity/repos/julia/src/interpreter.c:223
eval_stmt_value at /home/topolarity/repos/julia/src/interpreter.c:174 [inlined]
eval_body at /home/topolarity/repos/julia/src/interpreter.c:647
jl_interpret_toplevel_thunk at /home/topolarity/repos/julia/src/interpreter.c:787
jl_toplevel_eval_flex at /home/topolarity/repos/julia/src/toplevel.c:938
jl_toplevel_eval_flex at /home/topolarity/repos/julia/src/toplevel.c:881
ijl_toplevel_eval_in at /home/topolarity/repos/julia/src/toplevel.c:989
eval at ./boot.jl:398 [inlined]
eval_user_input at /home/topolarity/repos/julia/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:167
repl_backend_loop at /home/topolarity/repos/julia/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:263
#start_repl_backend#48 at /home/topolarity/repos/julia/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:248
start_repl_backend at /home/topolarity/repos/julia/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:245
_jl_invoke at /home/topolarity/repos/julia/src/gf.c:2911 [inlined]
ijl_apply_generic at /home/topolarity/repos/julia/src/gf.c:3093
#run_repl#61 at /home/topolarity/repos/julia/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:404
run_repl at /home/topolarity/repos/julia/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:390
unknown function (ip: 0x7ff2541076f9)
_jl_invoke at /home/topolarity/repos/julia/src/gf.c:2911 [inlined]
ijl_apply_generic at /home/topolarity/repos/julia/src/gf.c:3093
#1070 at ./client.jl:440
unknown function (ip: 0x7ff2541036d5)
_jl_invoke at /home/topolarity/repos/julia/src/gf.c:2911 [inlined]
ijl_apply_generic at /home/topolarity/repos/julia/src/gf.c:3093
jl_apply at /home/topolarity/repos/julia/src/julia.h:2002 [inlined]
jl_f__call_latest at /home/topolarity/repos/julia/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:899 [inlined]
invokelatest at ./essentials.jl:896 [inlined]
run_main_repl at ./client.jl:424
repl_main at ./client.jl:600 [inlined]
_start at ./client.jl:574
jfptr__start_58025 at /home/topolarity/repos/julia/usr/lib/julia/sys.so (unknown line)
_jl_invoke at /home/topolarity/repos/julia/src/gf.c:2911 [inlined]
ijl_apply_generic at /home/topolarity/repos/julia/src/gf.c:3093
jl_apply at /home/topolarity/repos/julia/src/julia.h:2002 [inlined]
true_main at /home/topolarity/repos/julia/src/jlapi.c:583
jl_repl_entrypoint at /home/topolarity/repos/julia/src/jlapi.c:735
main at /home/topolarity/repos/julia/cli/loader_exe.c:58
unknown function (ip: 0x7ff269035d8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at ./julia (unknown line)
Allocations: 6031575 (Pool: 6024682; Big: 6893); GC: 10
[1]    26083 IOT instruction  ./julia --project=tmp

The text was updated successfully, but these errors were encountered:

topolarity · 2023-10-24T22:39:32Z

Problem appears to be an bug with our inference/SSA-lowering:

3 2 ─── %2   = ϒ (kwargs@_2)::Core.PartialStruct(Base.Pairs{Symbol, Float64, Tuple{Symbol}, @NamedTuple{progress::Float64}}, Any[@NamedTuple{progress::Float64}, Core.Const((:progress,))])
  └──── %3   = $(Expr(:enter, #108)) 
... 
  106 ─ %131 = ϒ (%127)::@NamedTuple{progress::Float64} 
  └────        $(Expr(:leave, :(%3)))  
  107 ─        goto #110
  108 ┄ %134 = φᶜ (%2, %131)::Core.PartialStruct(Base.Pairs{Symbol, Float64, Tuple{Symbol}, @NamedTuple{progress::Float64}}, Any[@NamedTuple{progress::Float64}, Core.Const((:progress,))])

Notice that the second ϒ does not seem to be accounted for in the Phi-C node, despite appearing explicitly as an argument.

Either inference forgot to account for it or, more likely, inference is correctly noticing that the second Upsilon node is un-observable by the PhiC node (since it flows directly into a non-catch leave block), in which case the bug is that the slot2ssa pass is inserting this Upsilon node anyway.

simeonschaub · 2023-10-24T22:57:19Z

This looks very similar to the segfaults I was seeing in #51800, could that be the same issue? How did you get Julia to dump the ptr queue?

vtjnash · 2023-10-24T22:58:32Z

Sounds like a codegen bug, since it should not synthesize a store to a location that is incompatible with it

oscardssmith · 2023-10-24T23:04:35Z

Probably worth noting that this doesn't appear on either 1.9 or 1.10 so it is a somewhat recent regression.

gbaraldi · 2023-10-24T23:46:48Z

I think codegen just follows inference here

Keno · 2023-10-25T00:05:45Z

This is probably my recent try/catch PR that improves the precision for type information in the catch block - they were often inferred very imprecisely previously, so codegen didn't have to worry about it.

Keno · 2023-10-25T00:45:04Z

More standalone MRE:

may_error(b) = Base.inferencebarrier(b) && error()
function phic_type4()
   a = (;progress = "a")
   try
       may_error(false)
       let b = Base.inferencebarrier(true) ? (;progress = 1.0) : a
           a = b
       end
   catch
   end
   GC.gc()
   return a
end

In #51852, we are coercing a boxed `Union{@NamedTuple{progress::String}, @NamedTuple{progress::Float64}}` to `@NamedTuple{progress::String}` via convert_julia_type. This results in a jl_cgval_t that has a Vboxed that points to a boxed `@NamedTuple{progress::Float64}` but with a `@NamedTuple{progress::String}` type tag that the up upsilonnode code then tries to unbox into the typed PhiC slot. This ends up treating the Float64 as a pointer and crashing in GC. Avoid this by adding a runtime check that the converted value is actually compatible (we already had this kind of check for the non-boxed cases) and then making the unboxing runtime-conditional on the type of the union matching the type of the phic. Fixes #51852

In #51852, we are coercing a boxed `Union{@NamedTuple{progress::String}, @NamedTuple{progress::Float64}}` to `@NamedTuple{progress::String}` via convert_julia_type. This results in a jl_cgval_t that has a Vboxed that points to a boxed `@NamedTuple{progress::Float64}` but with a `@NamedTuple{progress::String}` type tag that the up upsilonnode code then tries to unbox into the typed PhiC slot. This ends up treating the Float64 as a pointer and crashing in GC. Avoid this by adding a runtime check that the converted value is actually compatible (we already had this kind of check for the non-boxed cases) and then making the unboxing runtime-conditional on the type of the union matching the type of the phic.

In #51852, we are coercing a boxed `Union{@NamedTuple{progress::String}, @NamedTuple{progress::Float64}}` to `@NamedTuple{progress::String}` via convert_julia_type. This results in a jl_cgval_t that has a Vboxed that points to a boxed `@NamedTuple{progress::Float64}` but with a `@NamedTuple{progress::String}` type tag that the up upsilonnode code then tries to unbox into the typed PhiC slot. This ends up treating the Float64 as a pointer and crashing in GC. Avoid this by adding a runtime check that the converted value is actually compatible (we already had this kind of check for the non-boxed cases) and then making the unboxing runtime-conditional on the type of the union matching the type of the phic. Fixes #51852

topolarity added bug Indicates an unexpected problem or unintended behavior GC Garbage collector labels Oct 24, 2023

oscardssmith added regression Regression in behavior compared to a previous version bisect wanted labels Oct 24, 2023

Keno self-assigned this Oct 25, 2023

Keno mentioned this issue Oct 25, 2023

Properly guard UpsilonNode unboxed store #51853

Merged

oscardssmith mentioned this issue Nov 5, 2023

Unpredictable segfault on master when multithreading with many allocations #52032

Open

oscardssmith closed this as completed in 3886803 Nov 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Segfault upon GC in function with try-catch block with type-unstable local variable #51852

Segfault upon GC in function with try-catch block with type-unstable local variable #51852

topolarity commented Oct 24, 2023

topolarity commented Oct 24, 2023 •

edited

Loading

simeonschaub commented Oct 24, 2023

vtjnash commented Oct 24, 2023

oscardssmith commented Oct 24, 2023

gbaraldi commented Oct 24, 2023

Keno commented Oct 25, 2023

Keno commented Oct 25, 2023

Segfault upon GC in function with try-catch block with type-unstable local variable #51852

Segfault upon GC in function with try-catch block with type-unstable local variable #51852

Comments

topolarity commented Oct 24, 2023

topolarity commented Oct 24, 2023 • edited Loading

simeonschaub commented Oct 24, 2023

vtjnash commented Oct 24, 2023

oscardssmith commented Oct 24, 2023

gbaraldi commented Oct 24, 2023

Keno commented Oct 25, 2023

Keno commented Oct 25, 2023

topolarity commented Oct 24, 2023 •

edited

Loading