{lang}[GCCcore/9.3.0] Julia v1.5.2

[bartoldeman]

(created using eb --new-pr)

[bartoldeman]

This easyconfig is like #11180 except it builds its own openblas and suitesparse (using ILP64 so 64-bit integers). This avoids issues with binary packages such as Arpack.

[bartoldeman]

@boegelbot please test @ generoso

[boegelbot]

@bartoldeman: Request for testing this PR well received on generoso

PR test command 'EB_PR=11522 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_11522 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

• exit code: 0
• output:

Submitted batch job 8152

Test results coming soon (I hope)...

- notification for comment with ID 710275418 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[bartoldeman]

Test report by @bartoldeman
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
build-node.computecanada.ca - Linux centos linux 7.8.2003, x86_64, Intel Xeon Processor (Skylake, IBRS), Python 3.7.7
See https://gist.github.com/c436f4cdcf0b1ca0fbebc68ce3f28b19 for a full test report.

[boegelbot]

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
generoso-x-2 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/190eba9a1323ddad2e07724154f62003 for a full test report.

[verdurin]

Test report by @verdurin
SUCCESS
Build succeeded for 2 out of 2 (1 easyconfigs in this PR)
nuc.lan - Linux Fedora 32, x86_64, Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz, Python 3.8.5
See https://gist.github.com/799582b715429fa1a2374feb311ad443 for a full test report.

[smoors]

Test report by @smoors
SUCCESS
Build succeeded for 2 out of 2 (1 easyconfigs in total)
node201.hydra.os - Linux centos linux 7.7.1908, x86_64, Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz (broadwell), Python 2.7.5
See https://gist.github.com/ed40f3a71d06f2a324da4c26cb6103c5 for a full test report.

[boegel]

Don't merge this just yet please, I think we need a custom easyblock for Julia + support for installing Julia packages as extensions.

I think @zarybnicky is working on that currently?

[zarybnicky]

@boegel @smoors Yup, that I am, I started looking into it this morning.

[vchuravy]

If there are any questions about the build-system and how to get it to behave don't hesitate to ask me directly.

[bartoldeman]

One of the main issues we had in the past was the use of Arpack often as a somewhat obscure dependency of Distributions.jl. This is described here: https://www.d.i10o.ca/writing/julia-arpack-dummy/

However since JuliaStats/PDMats.jl#115 the PDMats (intermediate dep) no longer depends on Arpack so the issue will probably no longer occur as often.

We had a workaround in place in building our own Arpack instead of what it normally does (downloading precompiled binaries depending on the GCC version used)

This went as follows:

# install arpack, overriding downloaded library...
postinstallcmds = [
   "JULIA_DEPOT_PATH=%(installdir)s/share/julia %(installdir)s/bin/julia -e 'using Pkg; Pkg.activate(\"distro\",shared=true); Pkg.add(\"Arpack\")' &> /dev/null",
   "sed -i 's/^    install/    #install/' %(installdir)s/share/julia/packages/Arpack/*/deps/build.jl",
   "module load arpack-ng/3.5.0 && ln -sf $EBROOTARPACKMINNG/lib/libarpack.so.2.0.0 %(installdir)s/share/julia/packages/Arpack/*/deps/usr/lib",
   "JULIA_DEPOT_PATH=%(installdir)s/share/julia %(installdir)s/bin/julia -e 'using Pkg; Pkg.activate(\"distro\",shared=true); Pkg.build(\"Arpack\"); using Arpack'",
   "rm -rf %(installdir)s/share/julia/registries",
]

but it's fragile and users' own depots and versions needed quickly got out of sync, so it ended up downloading Arpack anyway.
So best to be compatible in the end and not fight the (albeit not ideal from our point of view) system.

[bartoldeman]

Here's another useful bit of info from somebody at CSCS:
https://discourse.julialang.org/t/how-does-one-set-up-a-centralized-julia-installation/13922/31

[bartoldeman]

@victorusu ^^ maybe you can involve him?

[verdurin]

@bartoldeman any update on this one?

[bartoldeman]

@boegel told me to wait for @zarybnicky, I'm not sure if this should be abandoned or kept for reference for now.

[robert-mijakovic]

@zarybnicky @boegel @verdurin any news on this one? I have prepared an easyconfig for binary distribution of 1.5.3 with GCCcore/10.2.0. However, if we want to always compile from source, I can provide one based on @bartoldeman one.

[zarybnicky]

@robert-mijakovic My apologies on this one, I got stuck trying to match Pkg.jl with EasyBuild a la PythonPackage+Bundle, and then spent some time fighting health problems...
Compiling from source is preferable if at all realistic - by all means, if you have the time then go ahead :)

[bartoldeman]

This can still serve as a starting point for a newer version but 1.5.2 is too old now.