^(big(foo), bar) can cause SIGBART #8286

danluu · 2014-09-09T17:03:19Z

Example:

julia>^(big(96608869069402268615522366320733234710),16374500563449903721)
gmp: overflow in mpz type

signal (6): Aborted
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 1638697868)
__gmpz_n_pow_ui at /home/dluu/dev/julia/usr/bin/../lib/libgmp.so (unknown line)
^ at gmp.jl:337
bigint_pow at gmp.jl:346
^ at gmp.jl:351
jlcall_^;60129 at  (unknown line)
jl_apply at /home/dluu/dev/julia/src/julia.h:988
jl_trampoline at /home/dluu/dev/julia/src/builtins.c:821
jl_apply at /home/dluu/dev/julia/src/julia.h:988
jl_apply_generic at /home/dluu/dev/julia/src/gf.c:1592
unknown function (ip: 2066563356)
unknown function (ip: 2066564620)
unknown function (ip: 2066567534)
unknown function (ip: 2066563698)
unknown function (ip: 2066658267)
jl_toplevel_eval at /home/dluu/dev/julia/usr/bin/../lib/libjulia-debug.so (unknown line)
jl_f_top_eval at /home/dluu/dev/julia/src/builtins.c:425
eval_user_input at REPL.jl:54
jlcall_eval_user_input;60109 at  (unknown line)
jl_apply at /home/dluu/dev/julia/src/julia.h:988
jl_trampoline at /home/dluu/dev/julia/src/builtins.c:821
jl_apply at /home/dluu/dev/julia/src/julia.h:988
jl_apply_generic at /home/dluu/dev/julia/src/gf.c:1592
anonymous at task.jl:96
jl_apply at /home/dluu/dev/julia/src/julia.h:988
jl_trampoline at /home/dluu/dev/julia/src/builtins.c:821
unknown function (ip: 2066609306)
unknown function (ip: 2066612155)
unknown function (ip: 2066610812)
unknown function (ip: 2066610910)
jl_handle_stack_switch at /home/dluu/dev/julia/usr/bin/../lib/libjulia-debug.so (unknown line)
julia_trampoline at /home/dluu/dev/julia/usr/bin/../lib/libjulia-debug.so (unknown line)
unknown function (ip: 4203809)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 4199369)
Aborted (core dumped)

This is probably not specific to ^, since this happens when gmp tries to rellaoc too much space:

  if (sizeof (mp_size_t) == sizeof (int))
    {
      if (UNLIKELY (new_alloc > ULONG_MAX / GMP_NUMB_BITS))
        {
          fprintf (stderr, "gmp: overflow in mpz type\n");
          abort ();
        }
    }
  else
    {
      if (UNLIKELY (new_alloc > INT_MAX))
        {
          fprintf (stderr, "gmp: overflow in mpz type\n");
          abort ();
        }
    }

It seems a bit impolite to crash julia because of this. What do people think is a good fix?

Before calling gmp, try to figure out of the calculation will result in trying to realloc to much space.
Catch SIGABRT and throw a julia error
Patch gmp to return a null pointer instead of throwing SIGABRT
???

I don't like 1 because it requires doing something different for each function (I think?), but 2 and 3 have their own downsides.

The text was updated successfully, but these errors were encountered:

JeffBezanson · 2014-09-16T04:01:16Z

That's galling. Libraries should not call abort. Even more surprising, __gmpz_pow_ui returns void, so it easily could have returned an error code instead.

ViralBShah · 2014-12-12T02:21:26Z

Have we reported this upstream? Would be nice to have a ticket number.

felixrehren · 2017-01-23T17:16:15Z

Running the above in 0.6.0-dev produces

julia> ^(big(96608869069402268615522366320733234710),16374500563449903721)
ERROR: OverflowError()
Stacktrace:
 [1] bigint_pow(::BigInt, ::Int128) at .\gmp.jl:435
 [2] ^(::BigInt, ::Int128) at .\gmp.jl:442

julia>

Seems fixed. Should we test for this, or close?

JeffBezanson · 2017-01-23T18:15:23Z

No, I'm sure there are still cases where GMP can internally overflow and call abort. We have a very minimal check that happens to catch that case (on 32-bit systems).

musm · 2017-08-19T05:03:22Z

For reference: https://stackoverflow.com/questions/3558684/avoiding-abort-in-libgmp
https://stackoverflow.com/questions/13328409/overflow-handling-in-gmp-pow there doesn't seem to be any great solutions to this, but maybe: https://stackoverflow.com/a/14245611/2344930 (e.g. solution 1, proposed by OP)

grep0 · 2018-08-30T00:48:20Z

Still there in 1.0:

julia> BigInt(3)^3^3^3
gmp: overflow in mpz type

signal (6): Aborted
in expression starting at no file:0
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
__gmpz_realloc at /home/mvr/julia-1.0.0/bin/../lib/julia/libgmp.so (unknown line)
__gmpz_n_pow_ui at /home/mvr/julia-1.0.0/bin/../lib/julia/libgmp.so (unknown line)
pow_ui! at ./gmp.jl:146 [inlined]
pow_ui at ./gmp.jl:147

StefanKarpinski · 2018-08-30T01:26:57Z

Yes, it’s an upstream issue with GMP, which should not call abort like this. If someone wants to track down all the places GMP calls abort and fix them, it would be a worthy service.

maleadt · 2023-09-07T09:52:34Z

This regressed in 1.6:

❯ jl +1.5
julia> ^(big(96608869069402268615522366320733234710),16374500563449903721)
ERROR: OutOfMemoryError()
Stacktrace:
 [1] pow_ui! at ./gmp.jl:171 [inlined]
 [2] pow_ui(::BigInt, ::UInt64) at ./gmp.jl:172
 [3] ^ at ./gmp.jl:576 [inlined]
 [4] bigint_pow(::BigInt, ::Int128) at ./gmp.jl:597
 [5] ^(::BigInt, ::Int128) at ./gmp.jl:602
 [6] top-level scope at REPL[1]:1

❯ jl +1.6
julia> ^(big(96608869069402268615522366320733234710),16374500563449903721)
gmp: overflow in mpz type

signal (6): Aborted
in expression starting at REPL[1]:1
unknown function (ip: 0x7fbd3c08e83c)
raise at /usr/lib/libc.so.6 (unknown line)
abort at /usr/lib/libc.so.6 (unknown line)
__gmpz_n_pow_ui at /home/tim/Julia/depot/juliaup/julia-1.6.7+0.x64.linux.gnu/bin/../lib/julia/libgmp.so (unknown line)
pow_ui! at ./gmp.jl:171 [inlined]
pow_ui at ./gmp.jl:172
^ at ./gmp.jl:579 [inlined]
bigint_pow at ./gmp.jl:600
^ at ./gmp.jl:605

Even though we're still carrying the patch, and we do configure it:

julia> Base.GMP.ALLOC_OVERFLOW_FUNCTION[]
true

Other cases simply segfault now:

❯ jl +1.5 -e '555555555555555555555555555555555555555555555555555^55555555555555555'
ERROR: OutOfMemoryError()
Stacktrace:
 [1] pow_ui! at ./gmp.jl:171 [inlined]
 [2] pow_ui(::BigInt, ::UInt64) at ./gmp.jl:172
 [3] ^ at ./gmp.jl:576 [inlined]
 [4] bigint_pow(::BigInt, ::Int64) at ./gmp.jl:597
 [5] ^ at ./gmp.jl:602 [inlined]
 [6] macro expansion at ./none:0 [inlined]
 [7] literal_pow(::typeof(^), ::BigInt, ::Val{55555555555555555}) at ./none:0
 [8] top-level scope at none:1

❯ jl +1.6 -e '555555555555555555555555555555555555555555555555555^55555555555555555'
signal (11): Segmentation fault
in expression starting at none:1
__gmp_tmp_reentrant_alloc at /home/tim/Julia/depot/juliaup/julia-1.6.7+0.x64.linux.gnu/bin/../lib/julia/libgmp.so (unknown line)
__gmpz_n_pow_ui at /home/tim/Julia/depot/juliaup/julia-1.6.7+0.x64.linux.gnu/bin/../lib/julia/libgmp.so (unknown line)
pow_ui! at ./gmp.jl:171 [inlined]
pow_ui at ./gmp.jl:172
^ at ./gmp.jl:579 [inlined]
bigint_pow at ./gmp.jl:600
^ at ./gmp.jl:605 [inlined]
macro expansion at ./none:0 [inlined]
literal_pow at ./none:0

Bisected to #45375

Fixes #8286, which regressed due to a GMP upgrade in #45375.

Fixes #8286, which regressed due to a GMP upgrade in #45375. (cherry picked from commit 1097481)

JeffBezanson added the upstream The issue is with an upstream dependency, e.g. LLVM label Sep 16, 2014

jiahao mentioned this issue Jan 6, 2015

DomainError in bigint_pow #9618

Closed

JeffBezanson mentioned this issue Mar 1, 2016

gmp: overflow in mpz type #15293

Closed

JeffBezanson mentioned this issue Jun 17, 2016

Crash with << and BigInt: "gmp: overflow in mpz type" #16989

Closed

yuyichao mentioned this issue Aug 18, 2017

BigInt(2)^typemax(Culong) crashes julia #23340

Closed

grep0 mentioned this issue Sep 4, 2018

fix #8286: Throw OutOfMemoryError when gmp tries to allocate memory beyond the limit #29029

Closed

musm mentioned this issue Mar 4, 2019

rebase: Throw OutOfMemoryError when gmp tries to allocate memory beyond the limit #31215

Merged

KristofferC closed this as completed in #31215 Mar 6, 2019

thofma mentioned this issue Sep 5, 2019

Use patched GMP library from julia Nemocas/Nemo.jl#615

Closed

samuela mentioned this issue Mar 30, 2021

julia: init julia_16 NixOS/nixpkgs#118000

Closed

10 tasks

maleadt reopened this Sep 7, 2023

maleadt mentioned this issue Sep 8, 2023

GMP: Gracefully handle more overflows. #51243

Merged

maleadt closed this as completed in #51243 Sep 11, 2023

maleadt added a commit that referenced this issue Sep 11, 2023

GMP: Gracefully handle more overflows. (#51243)

1097481

Fixes #8286, which regressed due to a GMP upgrade in #45375.

KristofferC pushed a commit that referenced this issue Sep 15, 2023

GMP: Gracefully handle more overflows. (#51243)

c12e851

Fixes #8286, which regressed due to a GMP upgrade in #45375. (cherry picked from commit 1097481)

nalimilan pushed a commit that referenced this issue Nov 5, 2023

GMP: Gracefully handle more overflows. (#51243)

71d6798

Fixes #8286, which regressed due to a GMP upgrade in #45375. (cherry picked from commit 1097481)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

^(big(foo), bar) can cause SIGBART #8286

^(big(foo), bar) can cause SIGBART #8286

danluu commented Sep 9, 2014

JeffBezanson commented Sep 16, 2014

ViralBShah commented Dec 12, 2014

felixrehren commented Jan 23, 2017

JeffBezanson commented Jan 23, 2017

musm commented Aug 19, 2017 •

edited

Loading

grep0 commented Aug 30, 2018

StefanKarpinski commented Aug 30, 2018 •

edited

Loading

maleadt commented Sep 7, 2023 •

edited

Loading

^(big(foo), bar) can cause SIGBART #8286

^(big(foo), bar) can cause SIGBART #8286

Comments

danluu commented Sep 9, 2014

JeffBezanson commented Sep 16, 2014

ViralBShah commented Dec 12, 2014

felixrehren commented Jan 23, 2017

JeffBezanson commented Jan 23, 2017

musm commented Aug 19, 2017 • edited Loading

grep0 commented Aug 30, 2018

StefanKarpinski commented Aug 30, 2018 • edited Loading

maleadt commented Sep 7, 2023 • edited Loading

musm commented Aug 19, 2017 •

edited

Loading

StefanKarpinski commented Aug 30, 2018 •

edited

Loading

maleadt commented Sep 7, 2023 •

edited

Loading