get() fails when run from a new thread

[jonathan-conder-sm]

Versions:

• Julia 1.9.2
• HTTP 1.9.14
• MbedTLS 1.1.7
• Ubuntu 22.04 x86_64

I'm seeing an issue similar to #970, but for threads adopted after HTTP is imported.

Minimal example:

import HTTP

@show Threads.maxthreadid()
@show length(HTTP.URIs.uri_reference_regex)

example = Base.@cfunction(Cvoid, ()) do
    @show Threads.maxthreadid()
    @show length(HTTP.URIs.uri_reference_regex)

    HTTP.get("https://www.example.com")
    nothing
end

@threadcall example Cvoid ()

Output:

Threads.maxthreadid() = 1
length(HTTP.URIs.uri_reference_regex) = 1
Threads.maxthreadid() = 2
length(HTTP.URIs.uri_reference_regex) = 1
fatal: error thrown and no exception handler available.
AssertionError(msg="0 < tid <= v")
_length_assert at ~/.julia/packages/URIs/gpp9J/src/URIs.jl:694
access_threaded at ~/.julia/packages/URIs/gpp9J/src/URIs.jl:685
#parse_uri_reference#6 at ~/.julia/packages/URIs/gpp9J/src/URIs.jl:125
parse_uri_reference at ~/.julia/packages/URIs/gpp9J/src/URIs.jl:123 [inlined]
URI at ~/.julia/packages/URIs/gpp9J/src/URIs.jl:146 [inlined]
request_uri at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:556 [inlined]
#request#37 at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:457
_jl_invoke at /cache/build/default-amdci5-2/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-2/julialang/julia-release-1-dot-9/src/gf.c:2940
request at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:455
#request#20 at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:315 [inlined]
request at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:313 [inlined]
request at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:313 [inlined]
#get#38 at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:518 [inlined]
get at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:518 [inlined]
#3 at /tmp/http.jl:10
unknown function (ip: 0x7fb0900fd0fb)
#5 at ./threadcall.jl:34
unknown function (ip: 0x7fb0900ffd81)
uv__queue_work at /workspace/srcdir/libuv/src/threadpool.c:305
worker at /workspace/srcdir/libuv/src/threadpool.c:122
unknown function (ip: 0x7fb0d4be9b42)
unknown function (ip: 0x7fb0d4c7b9ff)

Minor caveat: the docs for @threadcall say

It is very important that the called function does not call back into Julia, as it will segfault.

However I don't think this is true since Julia 1.9:

Threads started outside the Julia runtime (e.g. from C or Java) can now become able to call into Julia code by calling jl_adopt_thread. This is done automatically when entering Julia code via cfunction or a @ccallable entry point. As a consequence, the number of threads can now change during execution.

In any case I initially observed this when calling from C, it's just harder to make a minimal example that way.