ntl: 9.11.0 -> 11.2.1 #43679
ntl: 9.11.0 -> 11.2.1 #43679
Conversation
|
Looks OK; I do not personally care about |
|
No attempt on x86_64-darwin (full log) The following builds were skipped because they don't evaluate on x86_64-darwin: ntl Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: ntl Partial log (click to expand)
|
|
Failure on x86_64-linux (full log) Attempted: ntl Partial log (click to expand)
|
ntl hasn't been updated in a while. So I'm doing that and adding myself as the maintainer. I'm also adding some options and pinning the sage dependency, since it is unfortunately not compatible with the latest ntl yet. I've also enabled the tests, since they don't take terribly long and are worth the time in my opinion.
Great! Can you merge this then? I think the less-than-optimal override situation is a different issue (although I'd gladly discuss options here). |
|
No attempt on x86_64-darwin (full log) The following builds were skipped because they don't evaluate on x86_64-darwin: ntl Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: ntl Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: ntl Partial log (click to expand)
|
|
Follow-up to the overlay issue in #43755. |
|
Looks like this introduces an issue: |
|
Illegal instruction? I have a bad feeling… |
|
Thanks for telling me (what happened to hydra mailing maintainers on failures?) I can imagine that it has something to do with the "tune" variable. I've only tested the x86 option and only on one cpu. Is it possible to determine which architecture that hydra runner is using? @GrahamcOfBorg build ntl |
|
Success on aarch64-linux (full log) Attempted: ntl Partial log (click to expand)
|
|
No attempt on x86_64-darwin (full log) The following builds were skipped because they don't evaluate on x86_64-darwin: ntl Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: ntl Partial log (click to expand)
|
|
@victorshoup (hoping that this is the correct github handle for the ntl author) As I recently emailed you, I'm trying to (well actually did) upgrade ntl for the nix package manager. Unfortunately our build servers are reporting an illegal instruciton during testing: While the build worked fine locally and on our CI. I'm using That build server is x86_64-linux. I think its an intel e5620, but I'm not sure about that. If you think its relevant I'll have to see if I can find out. The other configure flags I'm using are (with |
|
Hi,
So I imagine the problem is the following.
By default, NTL compiles with -march=native, so it will generate instructions
that may only be valid for the machine on which it is compiled.
You may have to configure with NATIVE=off to make the binary
work across different x86 architectures.
While that should work, you should note that many users who use these
binaries will be using very suboptimal builds.
I *strongly* encourage you to provide and encourage users to build NTL from source,
especially if they want truly high performance.
…-Victor
On Jul 19, 2018, at 4:59 PM, Timo Kaufmann ***@***.***> wrote:
@victorshoup (hoping that this is the correct github handle for the ntl author)
As I recently emailed you, I'm trying to (well actually did) upgrade ntl for the nix package manager. Unfortunately our build servers are reporting an illegal instruciton during testing:
running tests...............make: *** [makefile:405: check] Illegal instruction
While the build worked fine locally and on our CI. I'm using TUNE=x86 in this case. The documentation says that that "chooses options that should be well suited for most x86 platforms". Could the "most" in that sentence be the cause for that error? Or can you imagine some other cause?
That build server is x86_64-linux. I think its an intel e5620, but I'm not sure about that. If you think its relevant I'll have to see if I can find out.
The other configure flags I'm using are (with $(out) and ${gf2x} expanded to the proper directories):
DEF_PREFIX=$(out)
SHARED=on
NATIVE=off
NTL_GF2X_LIB=on
GF2X_PREFIX=${gf2x}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
So I imagine the problem is the following.
By default, NTL compiles with -march=native, so it will generate instructions
that may only be valid for the machine on which it is compiled.
You may have to configure with NATIVE=off to make the binary
work across different x86 architectures.
I am using `NATIVE=off` (see the list of configure flags in my last post) and the tests are running on the same machine as the build. So I don't think that is the issue.
While that should work, you should note that many users who use these
binaries will be using very suboptimal builds.
I *strongly* encourage you to provide and encourage users to build NTL from source,
especially if they want truly high performance.
I do offer a `tune` option that will do that (and build from source). *Encouraging* to build from source kind of goes against the point of a package manager however (except if you are gentoo). And using tuning / non-native by default is especially against the point of nix (which tries to be reproducible).
But the option is there for people who need it.
|
|
Ok. You'll have to send me more details.
What's the drv file you sent me? I don't know what a drv file is.
That said, I really doubt this is an ntl issue. First, ntl itself does use a few inline asm instructions, but these are very, very old instruction types and have not changed in many years. So that means something is configured wrong in the compiler or in some of the libraries that ntl uses. For example, perhaps the problem is with gmp.
Anyway, send me detailed logs. You could also try running the test program that crashes in gdb and backtrace. That's probably the quickest way to figure out what's going on.
… On Jul 19, 2018, at 5:51 PM, Timo Kaufmann ***@***.***> wrote:
> So I imagine the problem is the following.
> By default, NTL compiles with -march=native, so it will generate instructions
> that may only be valid for the machine on which it is compiled.
>
> You may have to configure with NATIVE=off to make the binary
> work across different x86 architectures.
I am using `NATIVE=off` (see the list of configure flags in my last post) and the tests are running on the same machine as the build. So I don't think that is the issue.
> While that should work, you should note that many users who use these
> binaries will be using very suboptimal builds.
>
> I *strongly* encourage you to provide and encourage users to build NTL from source,
> especially if they want truly high performance.
I do offer a `tune` option that will do that (and build from source). *Encouraging* to build from source kind of goes against the point of a package manager however (except if you are gentoo). And using tuning / non-native by default is especially against the point of nix (which tries to be reproducible).
But the option is there for people who need it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
The naming of the file is a bit confusing, but that is the build log of ntl containing the failure.
That is possible. Its hard to debug right now because I cannot reproduce the problem locally and don't have direct access to the builder that had the problem. It is just one builder in our build cluster. I'll have to try to find someone that can reproduce the failure locally.
The build log is the file with the confusing name. Are there other logs that might be helpful?
Again the above issue :/ I'll try to find someone. @7c6f434c @matthewbauer did you try the build locally? |
|
I don't know how to open a dry file (and neither does my computer).
Could you send me a build log that is a plain text file?
Thanks.
… On Jul 20, 2018, at 8:00 AM, Timo Kaufmann ***@***.***> wrote:
Ok. You'll have to send me more details. What's the drv file you sent me? I don't know what a drv file is.
The naming of the file is a bit confusing, but that is the build log of ntl containing the failure.
That said, I really doubt this is an ntl issue. First, ntl itself does use a few inline asm instructions, but these are very, very old instruction types and have not changed in many years. So that means something is configured wrong in the compiler or in some of the libraries that ntl uses. For example, perhaps the problem is with gmp.
That is possible. Its hard to debug right now because I cannot reproduce the problem locally and don't have direct access to the builder that had the problem. It is just one builder in our build cluster.
I'll have to try to find someone that can reproduce the failure locally.
Anyway, send me detailed logs.
The build log is the file with the confusing name. Are there other logs that might be helpful?
You could also try running the test program that crashes in gdb and backtrace. That's probably the quickest way to figure out what's going on.
Again the above issue :/ I'll try to find someone.
@7c6f434c @matthewbauer did you try the build locally?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
That is a plain text file. It really shouldn't have the At least for me, the browser just opens it. Otherwise you should be able to open it with any text editor. |
|
You should be able to just rename it to |
|
OK. Firefox opened it for me (but Vim shows unprintable gobbledygook).
Anyway, everything looks fin, until the Illegal instruction.
I guess the only way forward is to
1) find the platform where this happens
2) instead of running "make check", run:
make QuickTest
gbb ./QuickTest
3) run it (with no args), and the do a backtrace from the Illegal instruction.
Like I said, in principle, this should not be an NTL issue.
But...never say never, as they say...
… On Jul 20, 2018, at 11:45 AM, Timo Kaufmann ***@***.***> wrote:
That is a plain text file. It really shouldn't have the drv extension.
At least for me, the browser just opens it. Otherwise you should be able to open it with any text editor.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
OK. Firefox opened it for me (but Vim shows unprintable gobbledygook).
Oh, you're right. Weird. I don't know which black magic is involved here (right click -> save page on firefox works and vim opens that).
Anyway, everything looks fin, until the Illegal instruction.
I guess the only way forward is to
1) find the platform where this happens
2) instead of running "make check", run:
make QuickTest
gbb ./QuickTest
3) run it (with no args), and the do a backtrace from the Illegal instruction.
Looks like we can't do anything until someone can reproduce that error.
Maybe the gmp maintainers have an idea @peti @vrthra?
Like I said, in principle, this should not be an NTL issue.
But...never say never, as they say...
Thanks for helping trouble shoot it anyway :)
|
Motivation for this change
ntl hasn't been updated in a while. So I'm doing that and adding myself
as the maintainer. I'm also adding some options and pinning the sage
dependency, since it is unfortunately not compatible with the latest ntl
yet.
I've also enabled the tests, since they don't take terribly long and are
worth the time in my opinion.
@7c6f434c (for the flint change)
The overrides in sages
default.nixare really annoying to maintain (as seen in this patch). There has to be a better way than to "wire everything up" manually. Basically an overlay within nixpkgs.@matthewbauer
git logshows that you worked with overlays recently for the musl package set. Do you know if I could apply something similar for this purpose?I basically want to pin some of sages dependencies. But since there are so many dependencies, its important that those overrides apply recursively.
Things done
sandboxinnix.confon non-NixOS)nix-shell -p nox --run "nox-review wip"./result/bin/)nix path-info -Sbefore and after)