"realloc(): invalid pointer" running ForeignFunctions example in Ubuntu 18.04

[d-torrance]

After #3124 was merged, I'm getting the following, but only in Ubuntu 18.04:

i1 : needsPackage "ForeignFunctions";

i2 : x = mpzT 0;

i3 : mpzAdd = foreignFunction("__gmpz_add", void, {mpzT, mpzT, mpzT});

i4 : mpzAdd(x, 2, 3)
realloc(): invalid pointer
Aborted (core dumped)

[jkyang92]

Where is mpzT allocating memory the "limbs" of the integer (the actual array containing the data), the gc heap or the C heap. We don't replace the allocator for gmp, gmpz_add won't work if the limbs are in the gc heap.

[d-torrance]

Where is mpzT allocating memory the "limbs" of the integer (the actual array containing the data), the gc heap or the C heap. We don't replace the allocator for gmp, gmpz_add won't work if the limbs are in the gc heap.

Since #3124, the gc heap. That makes sense. I wonder why it works for Ubuntu post-18.04? And whichever macOS version the GitHub actions are running. Maybe I should revert that change before the release.

[d-torrance]

Ubuntu 18.04 still has GMP 6.1, and this item from the GMP 6.2 changelog looks possibly related:

Internal representation of the mpz_t variables now supports lazy allocation; memory is allocated only when a value is stored.

[d-torrance]

Yeah, I think the example in the documentation is small enough that later versions of GMP didn't actually allocate anything. But if we give it bigger numbers, the crash still happens. For example, in Ubuntu 22.04 using GMP 6.2.1:

i3 : x = mpzT 2^100

o3 = 1267650600228229401496703205376

o3 : ForeignObject of type mpz_t

i4 : mpzAdd(x, 3^100, 4^100)
realloc(): invalid pointer
Aborted (core dumped)

Process M2 exited abnormally with code 134

[jkyang92]

The only way to make this work directly is for mpzT to return a mpz_t that has a finalizer. A slightly more clever solution is to have a function that takes a function like mpzAdd and temporarily allocates an mpz_t for it to assign to and then convert it back for Macaualy2.

[d-torrance]

Yeah, that's basically what the previous behavior was. :) I'll revert the change soon.