Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include GMP 5.1.3 as an optional package #12661

Closed
simon-king-jena opened this issue Mar 13, 2012 · 53 comments
Closed

Include GMP 5.1.3 as an optional package #12661

simon-king-jena opened this issue Mar 13, 2012 · 53 comments

Comments

@simon-king-jena
Copy link
Member

Let's package GMP 5.1.3.

Upstream tarball at:

CC: @nexttime @rwst

Component: packages: optional

Keywords: gmp

Author: Simon King, Jean-Pierre Flori

Branch/Commit: a14aaf2

Reviewer: François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/12661

@simon-king-jena
Copy link
Member Author

comment:1

Google told me that some people suggested to "include cstdio to obtain std::FILE". So, I tried to add

#include <cstdio.h>

to gmp.h. However, after I attempted to build it, that line was gone!!

Can someone explain how I can avoid that my editing the file is reverted? Perhaps I need to do my change in the "patches" directory?

@simon-king-jena
Copy link
Member Author

comment:2

Yeah, I made some progress!!

I found that I needed to edit src/gmp-h.in, and I needed to #include <cstdio>, but only #if defined (__cplusplus).

Then, building the package went beyond the point where it failed previously. However, it turns out that the gmp package requires yacc. I am insalling it now.

TODO (for the moment):

  • Turn my changes to src/gmp-h.in into a patch, to be put into the patches directory.
  • spkg-install should check for the presence of yacc.

@simon-king-jena
Copy link
Member Author

comment:3

Hooray, the modified package builds on my openSuse laptop. However, spkg-check fails, ending with

Making check in cxx
make[3]: Entering directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
make  t-assign t-binary t-cast t-constr t-headers t-istream t-locale t-misc t-ops t-ostream t-prec t-rand t-ternary t-unary
make[4]: Entering directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-assign.o t-assign.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-assign  t-assign.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
mkdir .libs
g++ -O2 -m64 -mtune=k8 -o .libs/t-assign t-assign.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-assign
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-binary.o t-binary.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-binary  t-binary.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-binary t-binary.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-binary
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-cast.o t-cast.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-cast  t-cast.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-cast t-cast.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-cast
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-constr.o t-constr.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-constr  t-constr.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-constr t-constr.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-constr
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-headers.o t-headers.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-headers  t-headers.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-headers t-headers.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-headers
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-istream.o t-istream.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-istream  t-istream.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-istream t-istream.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-istream
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-locale.o t-locale.cc
t-locale.cc: In function ‘void check_input()’:
t-locale.cc:108:26: error: ‘abort’ was not declared in this scope
t-locale.cc:123:26: error: ‘abort’ was not declared in this scope
t-locale.cc: In function ‘void check_output()’:
t-locale.cc:158:18: error: ‘abort’ was not declared in this scope
make[4]: *** [t-locale.o] Fehler 1
make[4]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
make[3]: *** [check-am] Fehler 2
make[3]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
make[2]: *** [check-recursive] Fehler 1
make[2]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests'
make[1]: *** [check-recursive] Fehler 1
make[1]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src'
make: *** [check] Fehler 2

@simon-king-jena
Copy link
Member Author

comment:4

While I was at it, I tried to upgrade to gmp 5.0.4. Unfortunately, it complains that it can not find bdivmod.c in the path -- the old spkg contains that file in src/mpn/generic, but the original gmp 5.0.4 sources do no contain bdivmod.c in src/mpn/generic. Too bad.

@simon-king-jena
Copy link
Member Author

comment:5

The advantage of gmp 5.0.4 would be that the sources already include cstdio. So, no need for patching it. The problem is that 5.0.4 has removed and renamed a couple of things. For example, all files starting with the name mullow_ are now starting with mullo_. The patches of the spkg need to be changed accordingly.

The gmp 5.0.4 package seems to build. Keeping my fingers crossed for the test...

@simon-king-jena
Copy link
Member Author

comment:6

Replying to @simon-king-jena:

The gmp 5.0.4 package seems to build.

No, it doesn't. I get an infinite loop during configuration, it seems. /bin/sh ./config.status --recheck is repeated over and over again.

@simon-king-jena

This comment has been minimized.

@simon-king-jena
Copy link
Member Author

Author: Simon King

@simon-king-jena
Copy link
Member Author

Changed keywords from none to gmp

@simon-king-jena simon-king-jena changed the title Fix optional gmp package on openSuse Upgrade the optional GMP package Mar 13, 2012
@simon-king-jena
Copy link
Member Author

comment:8

See modified ticket description: It was easy to make gmp 5.0.4 build and pass all tests. Simply one has to drop all patches of the old gmp spkg.

@simon-king-jena
Copy link
Member Author

comment:9

A related ticket: #12661 (optional CLooG spkg).

CLooG needs gmp to build.

@simon-king-jena
Copy link
Member Author

comment:10

Replying to @simon-king-jena:

CLooG needs gmp to build.

Leif claims that CLooG could also be built with MPIR rather than GMP.

Anyway. Since the current spkg is broken (at least on openSuse), I still think we should upgrade. But people need to test on different platforms.

@simon-king-jena

This comment has been minimized.

@simon-king-jena

This comment has been minimized.

@nexttime
Copy link
Mannequin

nexttime mannequin commented Mar 15, 2012

comment:13

FWIW, I did build Sage against GMP (5.0.1 or 5.0.2) a couple of times in 2010 (when we made the "first big PARI upgrade") and never ran into any problems, except for the Lcalc spkg, which was the only one explicitly using -lmpir instead of -lgmp, an issue which AFAIK meanwhile is fixed.

Although I looked at the (btw. completely outdated) optional GMP spkg (on other occasions as well), I didn't use any of it, in particular didn't apply any additional patches to the more recent GMP versions.


We'll probably only run into trouble with using GMP instead of MPIR the day we upgrade FLINT, as newer (2.x, but IIRC only the most recent) versions of FLINT use some MPIR internals GMP doesn't provide, or at least isn't compatible with.

But for the moment it should be ok to use GMP instead, although (for CLooG) this shouldn't be necessary.

(In contrast, as Dima recently mentioned on sage-devel and mpir-devel, GAP uses some GMP internals MPIR currently doesn't provide. The [older?] version of GAP we use apparently works with MPIR... :-) )

@simon-king-jena
Copy link
Member Author

comment:14

Replying to @nexttime:

Although I looked at the (btw. completely outdated) optional GMP spkg ...

Yes, that's what this ticket is about.

in particular didn't apply any additional patches to the more recent GMP versions.

I'm not using any patches either. The question is whether other platforms (Solaris? OS X?) need patches.

We'll probably only run into trouble with using GMP instead of MPIR the day we upgrade FLINT,

When I first tried to build the optional gcc from #12369 with graphite (hence, together with the optional cloog-ppl package from #12666), I thought that cloog-ppl needs GMP (not MPIR). This is why I opened this ticket in the first place.

I found a massive slow-down when I built sage with the optimizations provided by graphite. Big surprise. But some people suggested that the culprit might in fact be GMP. So, I am started over. I can not tell yet whether Sage is faster with MPIR than with GMP. However, GMP is definitely not required by cloog-ppl, which lessens the priority of this ticket considerably, from my perspective...

@benjaminfjones
Copy link
Contributor

comment:15

Building the new gmp spkg here on Mac OS X (with gcc 4.6.3) with SAGE_CHECK=yes succeeds and passess all tests!

@nexttime
Copy link
Mannequin

nexttime mannequin commented Jul 28, 2012

comment:16

GMP 5.0.5 is out since a while...

@simon-king-jena
Copy link
Member Author

comment:17

Replying to @nexttime:

GMP 5.0.5 is out since a while...

This ticket is open since a while...

@nexttime
Copy link
Mannequin

nexttime mannequin commented Jul 28, 2012

comment:18

Replying to @simon-king-jena:

Replying to @nexttime:

GMP 5.0.5 is out since a while...

This ticket is open since a while...

I was wanting to review it since a while...

@jdemeyer jdemeyer modified the milestones: sage-5.11, sage-5.12 Aug 13, 2013
@kiwifb
Copy link
Member

kiwifb commented Dec 26, 2013

comment:20

gmp 5.1.x is out and transition to git is needed if we want to keep gmp in sage.

@jpflori
Copy link

jpflori commented Jan 3, 2014

comment:21

Yup, also note that FLINT >= 2.4 can now be built on top of GMP.
IIRC it's the only spkg which explicitely depended on MPIR.

@jpflori
Copy link

jpflori commented Feb 27, 2014

comment:30

Similar probledm and solution here:

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Feb 27, 2014

Changed commit from 2c34755 to a14aaf2

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Feb 27, 2014

Branch pushed to git repo; I updated commit sha1. New commits:

a14aaf2Explicit conversion in farey.cpp as recent GMP do not do it automatically.

@kiwifb
Copy link
Member

kiwifb commented Feb 27, 2014

comment:32

Replying to @jpflori:

Anyway, it also fails with CXX:

g++ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include/python2.7 -c sage/modular/arithgroup/farey.cpp -o build/temp.linux-ppc64-2.7/sage/modular/arithgroup/farey.o -fno-strict-aliasing -w
sage/modular/arithgroup/farey.cpp: In function ‘mpz_class floor(mpq_class)’:
sage/modular/arithgroup/farey.cpp:147:44: error: conversion from ‘mpq_class {aka __gmp_expr<__mpq_struct [1], __mpq_struct [1]>}’ to non-scalar type ‘mpz_class {aka __gmp_expr<__mpz_struct [1], __mpz_struct [1]>}’ requested

Sorry we could have warned you about that since sage-on-gentoo is stuck on gmp we knew about this and how to fix it.

@jpflori
Copy link

jpflori commented Feb 28, 2014

comment:33

Replying to @kiwifb:

Replying to @jpflori:

Anyway, it also fails with CXX:

g++ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include/python2.7 -c sage/modular/arithgroup/farey.cpp -o build/temp.linux-ppc64-2.7/sage/modular/arithgroup/farey.o -fno-strict-aliasing -w
sage/modular/arithgroup/farey.cpp: In function ‘mpz_class floor(mpq_class)’:
sage/modular/arithgroup/farey.cpp:147:44: error: conversion from ‘mpq_class {aka __gmp_expr<__mpq_struct [1], __mpq_struct [1]>}’ to non-scalar type ‘mpz_class {aka __gmp_expr<__mpz_struct [1], __mpz_struct [1]>}’ requested

Sorry we could have warned you about that since sage-on-gentoo is stuck on gmp we knew about this and how to fix it.

No problem, I found the solution more than easily on the web.

Any other suggestion from sage-on-gentoo?
E.g., do you have devised any simple way to switch from gmp to mpir and back again easily?
Or are you really stuck with gmp and cannot use mpir? (that would be strange.)

@kiwifb
Copy link
Member

kiwifb commented Feb 28, 2014

comment:34

We are really stuck with gmp unless we make great effort. It is complicated because most of the package of the tree will depend on gmp and few will use mpir directly, at some point I tried to port a lot of ebuild to mpir directly but it was a bit unsustainable. The easiest way is to use the gmp compatibility layer from mpir. Then we have to convince the toolchain people that we should have the possibility of switching between gmp and mpir to build gcc. Then we need to replace all dependencies to gmp in the tree by some virtual package which can be either gmp or mpir passing as gmp. That's a ton of work, require convincing people and so on.

if all package in sage could be configured with either real gmp or real mpir out of the box we may be able to do some easy switching between the two but it is not the case.
lmona.de could go its own way and use mpir exclusively if they wanted too (by calling it gmp mainly).

@jpflori
Copy link

jpflori commented Feb 28, 2014

comment:35

FTR here are the only failing doctests:

sage -t --long src/sage/rings/integer.pyx
**********************************************************************
File "src/sage/rings/integer.pyx", line 1955, in sage.rings.integer.Integer.__pow__
Failed example:
    2^(2^63-2)
Expected:
    Traceback (most recent call last):
    ...
    MemoryError: failed to allocate 1152921504606847008 bytes
Got:
    gmp: overflow in mpz type   
    Traceback (most recent call last):
      File "/home/jpflori/sage.git/local/lib/python2.7/site-packages/sage/doctest/fo
rker.py", line 480, in _run
        self.execute(example, compiled, test.globs)
      File "/home/jpflori/sage.git/local/lib/python2.7/site-packages/sage/doctest/fo
rker.py", line 839, in execute  
        exec compiled in globs  
      File "<doctest sage.rings.integer.Integer.__pow__[27]>", line 1, in <module>
        Integer(2)**(Integer(2)**Integer(63)-Integer(2))
      File "integer.pyx", line 1994, in sage.rings.integer.Integer.__pow__ (sage/rin
gs/integer.c:14075)
      File "c_lib.pyx", line 85, in sage.ext.c_lib.sig_raise_exception (sage/ext/c_l
ib.c:1009)
    RuntimeError: Aborted
**********************************************************************
File "src/sage/rings/integer.pyx", line 5401, in sage.rings.integer.Integer._xgcd
Failed example:
    2._xgcd(-2)
Expected:
    (2, 1, 0)
Got:
    (2, 0, -1)
**********************************************************************

and

sage -t --long src/sage/tests/book_stein_modform.py
**********************************************************************
File "src/sage/tests/book_stein_modform.py", line 94, in sage.tests.book_stein_modform
Failed example:
    [x.lift_to_sl2z(2) for x in M.manin_generators()]
Expected:
    [[1, 0, 0, 1], [0, -1, 1, 0], [0, -1, 1, 1]]
Got:
    [[1, 0, 0, 1], [0, -1, 1, 0], [1, 0, 1, 1]]
**********************************************************************

@kiwifb
Copy link
Member

kiwifb commented Feb 28, 2014

comment:36

I got the first one in sage/rings/integer.pyx and knew it was from gmp vs mpir. I also have sage/tests/book_stein_modform.py but I didn't know it was originating from gmp vs mpir. I don't have the second failure in integer.pyx it may be because I am currently only at gmp 5.1.2.

@jpflori
Copy link

jpflori commented Feb 28, 2014

comment:37

The ones in integer.pyx are really harmless anyway.

Not sure about the one in the Manin generators computation, so I could be easily convinced it is innocuous as well.

@kiwifb
Copy link
Member

kiwifb commented Feb 28, 2014

comment:38

Actually I got the second one in integer.pyx. It wasn't present when I last made a record of doctests failures in sage-on-gentoo for sage 6.0, but it is in 6.2.beta2.

@kiwifb
Copy link
Member

kiwifb commented Mar 15, 2014

comment:39

Any reason why this is still stuck at "need work"? I can give this a positive review, fixing the doctests to work with gmp is probably more work than it is worth. Especially the top one in integer.pyx.

Opinions?

@jpflori
Copy link

jpflori commented Mar 17, 2014

comment:40

I agree that I can live with the failing doctests.

As far as setting this to "needs_review" is concerned, I did not change the status because I wanted to include some patches to let SAge actually use GMP rather than MPIR and even potentially being able to switch from one to the other.

@kiwifb
Copy link
Member

kiwifb commented Mar 17, 2014

comment:41

The only thing I can see right now that prevent sage to build with gmp out of the box is the inclusion of mpir.h instead of gmp.h in memory.c in clib.

@jpflori
Copy link

jpflori commented Mar 17, 2014

comment:42

Flint install script should be modified as well.
Ok maybe it works without modification but passing with-gmp is better than with-mpir now that it is supported.

@jpflori
Copy link

jpflori commented Mar 17, 2014

comment:43

And you should also prevent Sage from building mpir I guess.

@kiwifb
Copy link
Member

kiwifb commented Mar 17, 2014

comment:44

Ambitious I see. Of course I have total control in sage-on-gentoo but I would appreciate just having to switch --with-gmp or something.

@jpflori
Copy link

jpflori commented Mar 17, 2014

comment:45

Sure.

I would be ok to postpone the proper way to use gmp instead of mpir for sage the distrib in a follow up ticket.
Let me dig up the few simple fixes needed for minimal support and push them here first.

@jpflori
Copy link

jpflori commented Mar 17, 2014

comment:46

Ok so everything is in fact already here.

@jpflori
Copy link

jpflori commented Mar 17, 2014

Changed author from Simon King to Simon King, Jean-Pierre Flori

@jpflori
Copy link

jpflori commented Mar 17, 2014

comment:47

Follow up at #15957.

@kiwifb
Copy link
Member

kiwifb commented Mar 17, 2014

Reviewer: François Bissey

@kiwifb
Copy link
Member

kiwifb commented Mar 17, 2014

comment:48

I don't know that the renaming of sage_mpir_* to sage_gmp_* is necessary. In fact I am not even sure it needs to be named gmp or mpir at all. Aside from that it is all ok and conform to what we do in sage-on-gentoo.
It also would cross a patch and a sed from my ebuilds at least.

@vbraun
Copy link
Member

vbraun commented Mar 20, 2014

Changed branch from u/jpflori/ticket/12661 to a14aaf2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants