tox.ini: Add raspbian-buster #29091

jaapspies · 2020-01-28T09:52:03Z

On Raspberry Pi 4 4GB on Raspbian Buster build failed with fplll:

libtool: link: g++ -std=gnu++11 -std=c++11 -fPIC -I/home/pi/sagemath/sage-9.1.be
ta1/local/include/ -O3 -Wl,-rpath -Wl,/home/pi/sagemath/sage-9.1.beta1/local/lib
 -o .libs/fplll main.o  -L/home/pi/sagemath/sage-9.1.beta1/local/lib -L/lib ./.l
ibs/libfplll.so -lmpfr -lgmp -pthread -Wl,-rpath -Wl,/home/pi/sagemath/sage-9.1.
beta1/local/lib
/usr/bin/ld: ./.libs/libfplll.so: undefined reference to `__atomic_store_8'
/usr/bin/ld: ./.libs/libfplll.so: undefined reference to `__atomic_load_8'
collect2: error: ld returned 1 exit status

This is fixed by upgrading FPLLL in #30021.

In this ticket, we add Docker-based testing for the platform raspbian-buster-armhf so that it can be verified that FPLLL now builds on this platform by the following:

  tox -e docker-raspbian-buster-armhf-standard -- fplll

Depends on #30021

Upstream: Completely fixed; Fix reported upstream

CC: @malb @slel @fchapoton @kliem

Component: build

Keywords: raspberry

Author: Matthias Koeppe

Branch/Commit: 1b4c497

Reviewer: Jaap Spies

Issue created by migration from https://trac.sagemath.org/ticket/29091

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-02-09T15:48:41Z

comment:1

Attachment: fplll-5.3.2.log

I have added raspbian-jessie and raspbian-stretch to #29143. There appears to be no Docker image for raspbian-buster.

mkoeppe · 2020-05-01T04:28:42Z

comment:2

Moving tickets to milestone sage-9.2 based on a review of last modification date, branch status, and severity.

fchapoton · 2020-05-13T11:22:15Z

Changed keywords from none to raspberry

malb · 2020-06-24T10:31:56Z

comment:4

This might be fixed upstream, can you try again? See fplll/fplll#418

jaapspies · 2020-06-24T13:55:36Z

comment:5

Replying to @malb:

This might be fixed upstream, can you try again? See fplll/fplll#418

Building sage-9.1 I get still the same ailure:

/bin/bash ../libtool  --tag=CXX   --mode=link g++ -std=gnu++11 -std=c++11  -fPIC -I/home/pi/sagemath/sage-9.1/local/include/ -L/home/pi/sagemath/sage-9.1/local/lib -O3  -Wl,-rpath-link,/home/pi/sagemath/sage-9.1/local/lib -L/home/pi/sagemath/sage-9.1/local/lib -Wl,-rpath,/home/pi/sagemath/sage-9.1/local/lib  -L/lib -o fplll main.o libfplll.la -lmpfr -lmpfr -lgmp -lgmp 
libtool: link: g++ -std=gnu++11 -std=c++11 -fPIC -I/home/pi/sagemath/sage-9.1/local/include/ -O3 -Wl,-rpath-link -Wl,/home/pi/sagemath/sage-9.1/local/lib -Wl,-rpath -Wl,/home/pi/sagemath/sage-9.1/local/lib -o .libs/fplll main.o  -L/home/pi/sagemath/sage-9.1/local/lib -L/lib ./.libs/libfplll.so -lmpfr -lgmp -pthread -Wl,-rpath -Wl,/home/pi/sagemath/sage-9.1/local/lib
/usr/bin/ld: ./.libs/libfplll.so: undefined reference to `__atomic_store_8'
/usr/bin/ld: ./.libs/libfplll.so: undefined reference to `__atomic_load_8'
collect2: error: ld returned 1 exit status
make[7]: *** [Makefile:1165: fplll] Error 1
make[7]: Leaving directory '/home/pi/sagemath/sage-9.1/local/var/tmp/sage/build/fplll-5.3.2/src/fplll'
make[6]: *** [Makefile:739: all] Error 2
make[6]: Leaving directory '/home/pi/sagemath/sage-9.1/local/var/tmp/sage/build/fplll-5.3.2/src/fplll'
make[5]: *** [Makefile:483: all-recursive] Error 1
make[5]: Leaving directory '/home/pi/sagemath/sage-9.1/local/var/tmp/sage/build/fplll-5.3.2/src'
make[4]: *** [Makefile:370: all] Error 2
make[4]: Leaving directory '/home/pi/sagemath/sage-9.1/local/var/tmp/sage/build/fplll-5.3.2/src'
********************************************************************************
Error building fplll-5.3.2
********************************************************************************

real	51m45.333s
user	67m10.859s
sys	1m31.680s
************************************************************************
Error installing package fplll-5.3.2
************************************************************************
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and including the log file
  /home/pi/sagemath/sage-9.1/logs/pkgs/fplll-5.3.2.log
Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/home/pi/sagemath/sage-9.1/local/var/tmp/sage/build/fplll-5.3.2 and type 'make' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
  (cd '/home/pi/sagemath/sage-9.1/local/var/tmp/sage/build/fplll-5.3.2' && '/home/pi/sagemath/sage-9.1/sage' --buildsh)
When you are done debugging, you can type "exit" to leave the subshell.
************************************************************************

malb · 2020-06-24T14:09:17Z

comment:6

Sorry that fix isn't in Sage (or an upstream release yet). I meant trying upstream master. I should cut a new release, though.

slel · 2020-06-24T16:31:13Z

Upstream: Reported upstream. Developers acknowledge bug.

slel · 2020-06-24T16:31:13Z

comment:7

I opened this issue in the fplll issue tracker

Fix ARM-specific problem with new multicore code in fplll 5.3.x

Fix ARM-specific problem with new multicore code in fplll 5.3.x fplll/fplll#432

but apparently this issue might have been relevant

add -latomic when needed

add -latomic when needed fplll/fplll#418

and it was fixed in Apr 2020 by this pull request

check for -latomic

check for -latomic fplll/fplll#419

The current latest release, fplll 5.3.2, is from Jan 2020.
Thus the fix should be tried by using the master branch.
I hope it fixes things and the next release works on ARM.
Sorry for the extra noise.

mkoeppe · 2020-06-24T20:21:09Z

Branch: u/mkoeppe/raspian-buster

mkoeppe · 2020-06-24T20:22:41Z

comment:9

I have just added a test environment for raspbian-buster.
You can try

tox -e docker-raspbian-buster-standard -- fplll

New commits:

`e5e2244`	`tox.ini, build/bin/write-dockerfile.sh: Add raspbian-buster`
`714bdce`	`.github/workflows/tox*.yml: Add raspbian-stretch, raspbian-buster`

mkoeppe · 2020-06-24T20:22:41Z

Commit: 714bdce

mkoeppe · 2020-06-24T20:23:19Z

comment:10

Tests running at https://github.com/mkoeppe/sage/actions/runs/146618564

mkoeppe · 2020-06-24T22:15:32Z

comment:11

Running this on my Mac, I was able to reproduce the error:

  [fplll-5.3.2] error installing, exit status 1. End of log file:
  [fplll-5.3.2]   make[5]: *** [Makefile:1175: latsieve] Error 1
  [fplll-5.3.2]   /usr/bin/ld: ./.libs/libfplll.so: undefined reference to `__atomic_store_8'
  [fplll-5.3.2]   /usr/bin/ld: ./.libs/libfplll.so: undefined reference to `__atomic_load_8'
  [fplll-5.3.2]   collect2: error: ld returned 1 exit status
  [fplll-5.3.2]   make[5]: *** [Makefile:1165: fplll] Error 1
  [fplll-5.3.2]   make[5]: Target 'all-am' not remade because of errors.
  [fplll-5.3.2]   make[5]: Leaving directory '/sage/local/var/tmp/sage/build/fplll-5.3.2/src/fplll'
  [fplll-5.3.2]   make[4]: *** [Makefile:739: all] Error 2

mkoeppe · 2020-06-25T00:08:07Z

Changed author from jsp to Matthias Koeppe

mkoeppe · 2020-06-25T04:37:29Z

comment:13

Replying to @mkoeppe:

I have just added a test environment for raspbian-buster.
You can try
tox -e docker-raspbian-buster-standard -- fplll

Actually, it is

  tox -e docker-raspbian-buster-armhf-standard -- fplll

which will also work on Linux Docker.

Tests with fplll@master run at https://github.com/mkoeppe/fplll/actions/runs/146958161

slel · 2020-06-25T13:06:57Z

comment:14

Replying to @mkoeppe:

Tests with fplll@master run at https://github.com/mkoeppe/fplll/actions/runs/146958161

It says:

The job running on runner GitHub Actions 36 has exceeded the maximum execution time of 360.

Could a tox environment install most dependencies using Conda via miniforge?

Miniforge provides Conda on ARM:

https://github.com/conda-forge/miniforge/

mkoeppe · 2020-06-25T17:15:51Z

comment:15

fplll@master on raspbian-buster-armhf-standard succeeded in 5h 7m 1s at
https://github.com/mkoeppe/fplll/runs/805921434

This confirms that master contains the correct fix.

mkoeppe · 2020-06-25T17:28:40Z

comment:16

fplll/fplll#433

mkoeppe · 2020-06-25T18:41:14Z

Changed upstream from Reported upstream. Developers acknowledge bug. to Fixed upstream, but not in a stable release.

sagetrac-git · 2020-06-26T00:10:14Z

Changed commit from 714bdce to e5e2244

sagetrac-git · 2020-06-26T00:10:14Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

mkoeppe · 2020-06-26T00:11:41Z

comment:19

Removed 714bdce from the branch - the build is so slow because of the emulation that it does not make sense to test the sage distribution on this platform on GH Actions.

malb · 2020-06-29T15:59:10Z

comment:20

See https://github.com/fplll/fplll/releases/tag/5.3.3 and #30021.

mkoeppe · 2020-06-29T16:31:54Z

Dependencies: #30021

sagetrac-git · 2020-07-03T04:30:07Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`242c552`	`update FPLLL to 5.3.3`
`53acc59`	`add upstream_url for fplll`
`1b4c497`	`tox.ini, build/bin/write-dockerfile.sh: Add raspbian-buster`

sagetrac-git · 2020-07-03T04:30:07Z

Changed commit from e5e2244 to 1b4c497

mkoeppe · 2020-07-03T04:30:32Z

Changed upstream from Fixed upstream, but not in a stable release. to Completely fixed; Fix reported upstream

jaapspies · 2020-07-09T14:06:56Z

comment:27

Works for me on a Raspberry Pi 4 4GB, Raspbian Buster

mkoeppe · 2020-07-09T14:12:51Z

comment:28

Here you can read more about how we address portability testing. One does not need an actual Raspberry Pi. https://doc.sagemath.org/html/en/developer/portability_testing.html#testing-sage-on-a-different-platform-using-docker

dimpase · 2020-07-09T14:58:31Z

comment:29

I noted on some other ticket that GitHub Actions offer an ARM platform.

vbraun · 2020-07-11T23:26:05Z

comment:30

Reviewer name is missing

slel · 2020-07-11T23:38:05Z

Reviewer: Jaap Spies

vbraun · 2020-07-15T22:25:15Z

Changed branch from u/mkoeppe/raspian-buster to 1b4c497

jaapspies added this to the sage-9.1 milestone Jan 28, 2020

jaapspies added c: build labels Jan 28, 2020

mkoeppe modified the milestones: sage-9.1, sage-9.2 May 1, 2020

This comment has been minimized.

Sign in to view

mkoeppe added s: needs review and removed p: major / 3 labels Jul 3, 2020

mkoeppe changed the title ~~Build failure of fplll-5.3.2 on Raspbian Buster (ARM)~~ tox.ini: Add raspbian-buster Jul 8, 2020

jaapspies added s: positive review and removed s: needs review labels Jul 9, 2020

vbraun added s: needs work and removed s: positive review labels Jul 11, 2020

slel added s: positive review and removed s: needs work labels Jul 11, 2020

vbraun removed the s: positive review label Jul 15, 2020

vbraun closed this as completed in 10f0dad Jul 15, 2020

tox.ini: Add raspbian-buster #29091

tox.ini: Add raspbian-buster #29091

Comments

jaapspies commented Jan 28, 2020

mkoeppe commented Feb 9, 2020

mkoeppe commented May 1, 2020

fchapoton commented May 13, 2020

malb commented Jun 24, 2020

jaapspies commented Jun 24, 2020

malb commented Jun 24, 2020

slel commented Jun 24, 2020

This comment has been minimized.

slel commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

mkoeppe commented Jun 25, 2020

mkoeppe commented Jun 25, 2020

slel commented Jun 25, 2020

mkoeppe commented Jun 25, 2020

mkoeppe commented Jun 25, 2020

mkoeppe commented Jun 25, 2020

sagetrac-git mannequin commented Jun 26, 2020

sagetrac-git mannequin commented Jun 26, 2020

mkoeppe commented Jun 26, 2020

malb commented Jun 29, 2020

mkoeppe commented Jun 29, 2020

sagetrac-git mannequin commented Jul 3, 2020

sagetrac-git mannequin commented Jul 3, 2020

mkoeppe commented Jul 3, 2020

This comment has been minimized.

jaapspies commented Jul 9, 2020

mkoeppe commented Jul 9, 2020

dimpase commented Jul 9, 2020

vbraun commented Jul 11, 2020

slel commented Jul 11, 2020

vbraun commented Jul 15, 2020