Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{lang}[foss/2020a] Julia v1.5.1 #11180

Closed
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions easybuild/easyconfigs/j/Julia/Julia-1.5.1-foss-2020a.eb
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
easyblock = 'ConfigureMake'

name = 'Julia'
version = '1.5.1'

homepage = 'https://julialang.org'
description = "Julia is a high-level, high-performance dynamic language for technical computing."

toolchain = {'name': 'foss', 'version': '2020a'}
toolchainopts = {'rpath': True}

source_urls = ['https://github.com/JuliaLang/julia/releases/download/v%(version)s/']
sources = ['julia-%(version)s-full.tar.gz']
checksums = ['1d0debfccfc7cd07047aa862dd2b1a96f7438932da1f5feff6c1033a63f9b1d4']

builddependencies = [
('Autotools', '20180311'),
('CMake', '3.16.4'),
('Python', '3.8.2'),
('Perl', '5.30.2'),
('pkg-config', '0.29.2'),
('PCRE2', '10.34'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is PCRE2 really a build dep? I suppose, maybe it statically links? But we use PCRE2 as a runtime dep in R for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw it listed in the Makefile as a build dep.

('patchelf', '0.12'),
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my incomplete attempt, I also marked gawk as required as a builddep for some reason. Maybe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jup, that's listed as a required build dep, but I didn't include it here, we should.


# some dependencies are provided through EasyBuild,
# but for others we let Julia build the ones included in the sources;
# adding SuiteSparse as a proper dep leads to problems with the tests,
# because some fail because they unset $LD_LIBRARY_PATH which leads to
# libraries like libamd.so not being found
dependencies = [
('GMP', '6.2.0'),
('MPFR', '4.0.2'),
('cURL', '7.69.1'),
('zlib', '1.2.11'),
('SuiteSparse', '5.7.1', '-METIS-5.1.0'),
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we list deps we leave to Julia to do it's own thing with? Based on my incomplete, broken, attempt, I identified:
LLVM
OpenLibm
utf8proc
libuv
libgit2
p7zip
dSFMT

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@Micket Micket Aug 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could probably could use these external as well if we want:

  • OpenLibm & utf8proc: No patches (though this is more or less just Julia specific anyway)
  • p7zip: Patches seem to CVE related, nothing julia specific
  • libunwind: Patches seem to be ARM specific
  • libgit2: Required patch to make ssh-agent problems non-fatal. Probably required, but I suppose it wouldn't hurt elsewhere, so we could possibly include that. It's been proposed for inclusion upstream (but without response from git devs).

Note: As Julia is transitioning more towards their BinaryBuilder tool where Julia modules can depend on pre-built "Artifacts", called "JLL"s, there is some potential conflict there if we try to control the versions to much and pulling in other libraries. Hard to say how it's all going to pan out.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Micket "libunwind: Patches seem to be ARM specific" is interesting. Could you expand on this?
We see Mesa and probably more stuff fail on arm due to libunwind: easybuilders/easybuild-easyblocks#2153. Is libunwind on arm something we can fix? I see arch and others ship it and use it for Mesa: https://archlinuxarm.org/packages/aarch64/mesa

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@terjekv This comment was just based on the patchset i found in julia: https://github.com/JuliaLang/julia/blob/master/deps/patches/ (loooks like a very very minor patch though). Also the yggdrasil package stuff: https://github.com/JuliaPackaging/Yggdrasil/tree/master/L/LibUnwind/bundled/patches. That's all i know.


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add osdeps for libssh2 (and mbedtls?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@boegel ping?

skipsteps = ['configure']

local_common_opts = "USE_BINARYBUILDER=0 "

# instruct Julia to use dependencies provided throuh EasyBuild
local_common_opts += 'USE_SYSTEM_BLAS=1 LIBBLAS="-L$EBROOTOPENBLAS/lib -lopenblas" LIBBLASNAME=libopenblas '
local_common_opts += 'USE_SYSTEM_LAPACK=1 LIBLAPACK="-L$EBROOTOPENBLAS/lib -lopenblas" LIBLAPACKNAME=libopenblas '
local_common_opts += "USE_SYSTEM_CURL=1 USE_SYSTEM_GMP=1 USE_SYSTEM_MPFR=1 USE_SYSTEM_PATCHELF=1 "
local_common_opts += "USE_SYSTEM_PCRE=1 USE_SYSTEM_ZLIB=1 "

buildopts = "release " + local_common_opts

# unset $TMPDIR to avoid failing test at test/file.jl:70
# instruct OpenBLAS to only use a single thread, some tests are sensitive to that;
pretestopts = "unset TMPDIR && export OPENBLAS_NUM_THREADS=1 && "

# running tests is disabled, because they are sensitive to a variety of things, including:
# * - having $TMPDIR defined makes some tests fail (see https://github.com/JuliaLang/julia/issues/35343);
# * - various tests use 'setenv' which resets the environment (incl. $LD_LIBRARY_PATH), which causes problems
# * with wrong libstdc++.so being picked up if commands are being run in the setenv. Similar problems
# arise with libraries provided by OpenBLAS, SuiteSparse, etc.
# (see https://github.com/JuliaLang/julia/issues/30192);
# * - some tests fail depending on number of available cores,
# or network config (see https://github.com/JuliaLang/julia/issues/36986);
runtest = False

# make sure dependencies provided by EasyBuild are used when testing,
# to avoid that patchelf is built from source, for example
testopts = local_common_opts

# specify installation prefix, and make sure dependencies provided by EasyBuild are used
installopts = "prefix=%(installdir)s " + local_common_opts

sanity_check_paths = {
'files': ['bin/julia', 'lib/libjulia.%s' % SHLIB_EXT],
'dirs': [],
}

sanity_check_commands = ["julia --help"]

moduleclass = 'lang'