Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Executing 'nix-hash' failed with status 1 #243

Closed
arraytools opened this issue Jul 8, 2024 · 31 comments
Closed

Executing 'nix-hash' failed with status 1 #243

arraytools opened this issue Jul 8, 2024 · 31 comments
Labels
bug Something isn't working

Comments

@arraytools
Copy link

Thanks for providing such a useful software. I have an issue when I try to follow the example on the vignette 'd1 - Installing R packages in a Nix environment' to install an old R package archived on CRAN. Any idea will be appreciated. Thanks.

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

library(rix)
path_default_nix <- tempdir()

rix(r_ver = "4.2.1",
r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
system_pkgs = NULL,
git_pkgs = NULL,
ide = "other",
project_path = path_default_nix,
overwrite = TRUE)
Error in sys::exec_internal(cmd = cmd, args = args) :
Executing 'nix-hash' failed with status 1

traceback()
17: stop(sprintf("Executing '%s' failed with status %d", cmd, status))
16: sys::exec_internal(cmd = cmd, args = args)
15: nix_sri_hash(path = path_to_source_root)
14: hash_url(url = repo_url)
13: hash_cran(repo_url = repo_url)
12: nix_hash(repo_url, commit)
11: get_sri_hash_deps(repo_url, branch_name = NULL, commit = NULL)
10: FUN(X[[i]], ...)
9: lapply(archive_pkgs, fetchzip)
8: paste(lapply(archive_pkgs, fetchzip), collapse = "\n")
7: fetchzips(archive_pkgs)
6: paste(fetchgits(git_pkgs), fetchzips(archive_pkgs), collapse = "\n")
5: fetchpkgs(git_pkgs, archive_pkgs)
4: sprintf("\n git_archive_pkgs = [%s ];\n", fetchpkgs(git_pkgs,
archive_pkgs))
3: generate_git_archived_pkgs(git_pkgs, cran_pkgs$archive_pkgs,
flag_git_archive)
2: paste(generate_header(nix_repo, r_ver, rix_call), generate_rpkgs(cran_pkgs$rPackages,
flag_rpkgs), generate_git_archived_pkgs(git_pkgs, cran_pkgs$archive_pkgs,
flag_git_archive), generate_tex_pkgs(tex_pkgs), generate_local_r_pkgs(local_r_pkgs,
flag_local_r_pkgs), generate_system_pkgs(system_pkgs, r_pkgs),
generate_wrapped_pkgs(ide, attrib, flag_git_archive, flag_rpkgs,
flag_local_r_pkgs), generate_shell(flag_git_archive,
flag_rpkgs, flag_tex_pkgs, flag_local_r_pkgs, flag_wrapper,
shell_hook), collapse = "\n")
1: rix(r_ver = "4.2.1", r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
system_pkgs = NULL, git_pkgs = NULL, ide = "other", project_path = path_default_nix,
overwrite = TRUE)

system("cat /etc/os-release")
PRETTY_NAME="KDE neon 6.0"
NAME="KDE neon"
VERSION_ID="22.04"
VERSION="6.0"
VERSION_CODENAME=jammy
ID=neon
ID_LIKE="ubuntu debian"
HOME_URL="https://neon.kde.org/"
SUPPORT_URL="https://neon.kde.org/"
BUG_REPORT_URL="https://bugs.kde.org/"
PRIVACY_POLICY_URL="https://kde.org/privacypolicy/"
UBUNTU_CODENAME=jammy
LOGO=start-here-kde-neon

@b-rodrigues
Copy link
Contributor

Thank you for reporting this, we have recently worked on nix-hash and it seems you've found a bug! Do you have nix installed? If not,this should be pulling the hash from a server we set up to compute the hash remotely. Maybe somethings wrong with the server: if you have Nix installed locally, the hash should be computed locally

@arraytools
Copy link
Author

Thanks for the quick reply. I installed nix via the command: sh <(curl -L https://nixos.org/nix/install) --daemon. Is it enough? I try several examples on the vignettes including no-versioned CRAN packages, Bioconductor packages, tar.gz local file, ... All works fine. But the versioned packages installation just don't work (i also tested it on Ubuntu 24.04 box).

@b-rodrigues
Copy link
Contributor

OK thank you for providing these details. There's something wrong with how versioned packages are pulled then. I'll try to get it fixed tomorrow!

@philipp-baumann
Copy link
Collaborator

philipp-baumann commented Jul 8, 2024

just a spontaneous thought, we are using the classical nix-hash but with relatively new syntax in Nix. @arraytools what version of nix are you using?

does this return anything if you run that on an arbitrary path in the shell (maybe rather lower level ;-) like a project directory?

# like here: https://github.com/b-rodrigues/rix/blob/699372eaadef9a24b7f23121641ce97754fc6659/R/nix_hash.R#L95-L99
# but now via wrapper and in shell
nix-hash --type sha256 --sri .

maybe this is too new? implemented here: NixOS/nix#7690

@arraytools
Copy link
Author

nix (Nix) 2.23.3

When I ran nix-hash --type sha256 --sri . inside a nix shell in a project directory, it returns
sha256-/D+Wumc1cr1ArJamF6Ab5UE3VVsD//yK+eW4dOiQAb0=

@philipp-baumann
Copy link
Collaborator

philipp-baumann commented Jul 8, 2024

nix (Nix) 2.23.3

When I ran nix-hash --type sha256 --sri . inside a nix shell in a project directory, it returns sha256-/D+Wumc1cr1ArJamF6Ab5UE3VVsD//yK+eW4dOiQAb0=

okey that's good to know this works. Could it be a permission issue?

Does it still not work when not using a tempdir?

path_default_nix <- file.path(".", "_test")

@arraytools
Copy link
Author

Same error :(

> path_default_nix <- file.path(".", "_test")
> rix(r_ver = "4.2.1",
    r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
    system_pkgs = NULL,
    git_pkgs = NULL,
    ide = "other",
    project_path = path_default_nix,
    overwrite = TRUE)
Error in sys::exec_internal(cmd = cmd, args = args) : 
  Executing 'nix-hash' failed with status 1

@philipp-baumann
Copy link
Collaborator

philipp-baumann commented Jul 8, 2024

ix(r_ver = "4.2.1",
r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
system_pkgs = NULL,
git_pkgs = NULL,
ide = "other",
project_path = path_default_nix,
overwrite = TRUE)

hmm. What's your sessionInfo()? wonder if you're running this in a nix-R session. Are you using nix-shell --pure?. Do you have any other ideas what could be buggy in our code. I wonder how this is related to how nix was installed, determinate systems approach vs. the classical.

@arraytools
Copy link
Author

I am running this in a regular R session (not inside a nix-shell environment). I also have a problem running the example in the "Installing packages from Github" section (same error message). Below is the sessionInfo output,

> sessionInfo()
R version 4.4.1 (2024-06-14)
Platform: x86_64-pc-linux-gnu
Running under: KDE neon 6.0

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/New_York
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rix_0.8.0

loaded via a namespace (and not attached):
[1] compiler_4.4.1   tools_4.4.1      curl_5.2.1       codetools_0.2-20
[5] jsonlite_1.8.8   sys_3.4.2 

Thanks.

@b-rodrigues
Copy link
Contributor

I cannot reproduce, neither from an R session from Nix, nor a "system" R session. I'm not on Ubuntu, but on openSuse, but I don't think it should matter much. Perhaps you aren't on the very latest rix version? We've had several commits with 0.8.0, maybe try to re-install rix.

@philipp-baumann
Copy link
Collaborator

can confirm, just tested via WSL2 ubuntu:

> path_default_nix <- file.path(".", "test")
> rix(r_ver = "4.2.1",
+     r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
    git_pkgs = NULL,
    ide = "other",
    project_path = path_default_nix,
+     system_pkgs = NULL,
+     git_pkgs = NULL,
+     ide = "other",
+     project_path = path_default_nix,
+     overwrite = TRUE
+ )
Error in rix(r_ver = "4.2.1", r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),  : 
  could not find function "rix"
> path_default_nix <- file.path(".", "test")
> rix::rix(r_ver = "4.2.1",
+     r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
    system_pkgs = NULL,
+     system_pkgs = NULL,
+     git_pkgs = NULL,
+     ide = "other",
+     project_path = path_default_nix,
+     overwrite = TRUE
+ )
Error in sys::exec_internal(cmd = cmd, args = args) : 
  Executing 'nix-hash' failed with status 1
> sessionInfo()
R version 4.3.3 (2024-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

time zone: Europe/Zurich
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.3   tools_4.3.3      curl_5.2.1       rix_0.8.0       
[5] codetools_0.2-18 jsonlite_1.8.8   sys_3.4.2

@b-rodrigues
Copy link
Contributor

is it an ubuntu specific thing ? but why does it pass on CI?

@philipp-baumann
Copy link
Collaborator

I don't have it in a nix-shell, but on a host session. it seems a GLIBC issue.

Browse[1]> system2(cmd, args)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by nix-hash)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/4z754a0vzl98asv0pa95i5d9szw5jqbs-lowdown-1.0.2-lib/lib/liblowdown.so.3)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixexpr.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/wqwxmdyadnz2nyk8jxfacwr0yph4bm2m-boehm-gc-8.2.2/lib/libgc.so.1)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixfetchers.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixstore.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixutil.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixcmd.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/8kg6s2b0ny3y0dh2xldh3sfxndg0qyhv-libgit2-20231018162940/lib/libgit2.so.1.8)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/71d6hngrqdl5zvn768fjlhmakmfi5c2w-aws-sdk-cpp-1.11.118/lib/libaws-cpp-sdk-core.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/1rkhjf55x59w6qm1pbhf80ks2wjpg973-libcpuid-0.6.4/lib/libcpuid.so.16)
Browse[1]> ls()
[1] "args"          "cmd"           "has_nix_shell" "path"         
Browse[1]> has_nix_shell
[1] TRUE
Browse[1]> sessionInfo()
R version 4.3.3 (2024-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

time zone: Europe/Zurich
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rix_0.8.0        testthat_3.2.1.1

loaded via a namespace (and not attached):
 [1] miniUI_0.1.1.1    jsonlite_1.8.8    compiler_4.3.3    brio_1.1.5       
 [5] promises_1.3.0    Rcpp_1.0.12       stringr_1.5.1     later_1.3.2      
 [9] fastmap_1.2.0     mime_0.12         R6_2.5.1          curl_5.2.1       
[13] htmlwidgets_1.6.4 desc_1.4.3        profvis_0.3.8     rprojroot_2.0.4  
[17] shiny_1.8.1.1     rlang_1.1.4       cachem_1.1.0      stringi_1.8.3    
[21] httpuv_1.6.15     fs_1.6.3          sys_3.4.2         pkgload_1.3.4    
[25] memoise_2.0.1     cli_3.6.3         withr_3.0.0       magrittr_2.0.3   
[29] digest_0.6.35     rstudioapi_0.16.0 xtable_1.8-4      remotes_2.5.0    
[33] devtools_2.4.5    lifecycle_1.0.4   vctrs_0.6.5       glue_1.7.0       
[37] urlchecker_1.0.1  sessioninfo_1.2.2 codetools_0.2-18  pkgbuild_1.4.4   
[41] purrr_1.0.2       usethis_2.2.3     tools_4.3.3       ellipsis_0.3.2   
[45] htmltools_0.5.8.1

@philipp-baumann
Copy link
Collaborator

is it an ubuntu specific thing ? but why does it pass on CI

is it an ubuntu specific thing ? but why does it pass on CI?

indeed a good question.

@b-rodrigues
Copy link
Contributor

I don't have it in a nix-shell, but on a host session. it seems a GLIBC issue.

we need to call rix_init before hashing perhaps?, for now it's called right at the end

@philipp-baumann
Copy link
Collaborator

I don't have it in a nix-shell, but on a host session. it seems a GLIBC issue.

we need to call rix_init before hashing perhaps?, for now it's called right at the end

Might be a hint. It looks for GLIBC from debian but it should't, so we get something related to: https://discourse.nixos.org/t/new-install-nix-wants-glibc-2-38-debian-12-has-2-36/36109

Browse[1]> ls()
[1] "args"          "cmd"           "has_nix_shell" "path"         
Browse[1]> has_nix_shell
[1] TRUE
Browse[1]> Sys.getenv("LD_LIBRARY_PATH")
[1] "/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/lib/server"

@philipp-baumann
Copy link
Collaborator

I don't have it in a nix-shell, but on a host session. it seems a GLIBC issue.

we need to call rix_init before hashing perhaps?, for now it's called right at the end

Might be a hint. It looks for GLIBC from debian but it should't, so we get something related to: https://discourse.nixos.org/t/new-install-nix-wants-glibc-2-38-debian-12-has-2-36/36109

Browse[1]> ls()
[1] "args"          "cmd"           "has_nix_shell" "path"         
Browse[1]> has_nix_shell
[1] TRUE
Browse[1]> Sys.getenv("LD_LIBRARY_PATH")
[1] "/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/lib/server"

I don't think its related. The .Rprofile is in path_default_nix (could be tempdir or not), and we call nix-hash directly via {sys}, so an .Rprofile would not have any effect there. Also tried to first put rix_init() does not change anything (same situation as there would be already an .Rprofile file.)

[1] "has_nix_shell"    "path"             "path_default_nix"
Browse[1]> system("nix-hash --type sha256 --sri /tmp/RtmpMfE35brepodghwt/package_src/dplyr")
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by nix-hash)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/4z754a0vzl98asv0pa95i5d9szw5jqbs-lowdown-1.0.2-lib/lib/liblowdown.so.3)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixexpr.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/wqwxmdyadnz2nyk8jxfacwr0yph4bm2m-boehm-gc-8.2.2/lib/libgc.so.1)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixfetchers.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixstore.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixutil.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixcmd.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/8kg6s2b0ny3y0dh2xldh3sfxndg0qyhv-libgit2-20231018162940/lib/libgit2.so.1.8)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/71d6hngrqdl5zvn768fjlhmakmfi5c2w-aws-sdk-cpp-1.11.118/lib/libaws-cpp-sdk-core.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/1rkhjf55x59w6qm1pbhf80ks2wjpg973-libcpuid-0.6.4/lib/libcpuid.so.16)
Browse[1]> path
[1] "/tmp/RtmpMfE35brepogjifw/package_src/dplyr"
Browse[1]> list.files("/tmp/RtmpMfE35brepogjifw/package_src/dplyr")
 [1] "build"       "data"        "DESCRIPTION" "inst"        "LICENSE"    
 [6] "man"         "MD5"         "NAMESPACE"   "NEWS.md"     "R"          
[11] "README.md"   "src"         "tests"       "vignettes"

@philipp-baumann
Copy link
Collaborator

Damn, there we go:

[1] "/tmp/RtmpMfE35brepogjifw/package_src/dplyr"
Browse[1]> list.files("/tmp/RtmpMfE35brepogjifw/package_src/dplyr")
 [1] "build"       "data"        "DESCRIPTION" "inst"        "LICENSE"    
 [6] "man"         "MD5"         "NAMESPACE"   "NEWS.md"     "R"          
[11] "README.md"   "src"         "tests"       "vignettes"  
Browse[1]> Sys.getenv("LD_LIBRARY_PATH")
[1] "/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/lib/server"
Browse[1]> Sys.setenv(LD_LIBRARY_PATH="")
Browse[1]> system("nix-hash --type sha256 --sri /tmp/RtmpMfE35brepodghwt/package_src/dplyr")
sha256-f30raalLd9KoZKZSxeTN71PG6BczXRIiP6g7EZeH09U=
Browse[1]> Sys.setenv(LD_LIBRARY_PATH="/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/lib/server")
Browse[1]> system("nix-hash --type sha256 --sri /tmp/RtmpMfE35brepodghwt/package_src/dplyr")
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by nix-hash)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/4z754a0vzl98asv0pa95i5d9szw5jqbs-lowdown-1.0.2-lib/lib/liblowdown.so.3)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixexpr.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/wqwxmdyadnz2nyk8jxfacwr0yph4bm2m-boehm-gc-8.2.2/lib/libgc.so.1)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixfetchers.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixstore.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixutil.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/00dc6iygj63mjcmixki1vwg7m1jbdi9p-nix-2.21.2/lib/libnixcmd.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/8kg6s2b0ny3y0dh2xldh3sfxndg0qyhv-libgit2-20231018162940/lib/libgit2.so.1.8)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/71d6hngrqdl5zvn768fjlhmakmfi5c2w-aws-sdk-cpp-1.11.118/lib/libaws-cpp-sdk-core.so)
nix-hash: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/1rkhjf55x59w6qm1pbhf80ks2wjpg973-libcpuid-0.6.4/lib/libcpuid.so.16)

@philipp-baumann
Copy link
Collaborator

philipp-baumann commented Jul 9, 2024

@arraytools what's your LD_LIBRARY_PATH in your R session that fails to run rix::rix() with hashing of GitHub and CRAN archive remotes? Does it resolve if you set it to "" in the current R session and then call the same rix::rix() command?

@b-rodrigues I implemented a workaround for Ubuntu for nix_build() here:

https://github.com/b-rodrigues/rix/blob/699372eaadef9a24b7f23121641ce97754fc6659/R/nix_build_helpers.R#L38-L61
low level and applying here:

https://github.com/b-rodrigues/rix/blob/699372eaadef9a24b7f23121641ce97754fc6659/R/nix_build.R#L42-L63

Why this happens? hmm, this seems a design flaw somewhere in Nix. In my opinion nix-shell should never link to any of system's glibc.

I feel it's a tweak we need to add to nix_hash() wrapper, too, and we should do that and and in the end not forget to revert to old LD_LIBRARY_PATH via on.exit() hook

@b-rodrigues
Copy link
Contributor

So we can apply the same fix as for nix_build ? that's good news!

@arraytools
Copy link
Author

When I set LD_LIBRARY_PATH to "" on the Ubuntu machine and rerun the example on vignettes, I'll get an error I attached below. This error can be repeated on another machine running Debian 11.

But if I use the current directory (or subdir) as the project path, it works. Same story goes to the example of installing packages from Github.

$ R

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> Sys.getenv("LD_LIBRARY_PATH")
[1] "/opt/R/4.4.1/lib/R/lib:/usr/local/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-11-openjdk-amd64/lib/server"
> Sys.setenv(LD_LIBRARY_PATH="")
> library(rix)
> path_default_nix <- tempdir()

rix(r_ver = "4.2.1",
    r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
    system_pkgs = NULL,
    git_pkgs = NULL,
    ide = "other",
    project_path = path_default_nix,
    overwrite = TRUE)
Error in file(con, "w") : cannot open the connection
In addition: Warning message:
In file(con, "w") :
  cannot open file '/tmp/RtmpOXyrO1/default.nix': No such file or directory
> traceback()
3: file(con, "w")
2: writeLines(default.nix, default.nix_path)
1: rix(r_ver = "4.2.1", r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"), 
       system_pkgs = NULL, git_pkgs = NULL, ide = "other", project_path = path_default_nix, 
       overwrite = TRUE)

Thank you for your help.

@philipp-baumann
Copy link
Collaborator

When I set LD_LIBRARY_PATH to "" on the Ubuntu machine and rerun the example on vignettes, I'll get an error I attached below. This error can be repeated on another machine running Debian 11.

But if I use the current directory (or subdir) as the project path, it works. Same story goes to the example of installing packages from Github.

$ R

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> Sys.getenv("LD_LIBRARY_PATH")
[1] "/opt/R/4.4.1/lib/R/lib:/usr/local/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-11-openjdk-amd64/lib/server"
> Sys.setenv(LD_LIBRARY_PATH="")
> library(rix)
> path_default_nix <- tempdir()

rix(r_ver = "4.2.1",
    r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
    system_pkgs = NULL,
    git_pkgs = NULL,
    ide = "other",
    project_path = path_default_nix,
    overwrite = TRUE)
Error in file(con, "w") : cannot open the connection
In addition: Warning message:
In file(con, "w") :
  cannot open file '/tmp/RtmpOXyrO1/default.nix': No such file or directory
> traceback()
3: file(con, "w")
2: writeLines(default.nix, default.nix_path)
1: rix(r_ver = "4.2.1", r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"), 
       system_pkgs = NULL, git_pkgs = NULL, ide = "other", project_path = path_default_nix, 
       overwrite = TRUE)

Thank you for your help.

Thanks @arraytools for reporting. So these are indeed separate issues. I can confirm the tempfile issue occurs on MacOS as well. Let's open a second ticket for this one.

@philipp-baumann
Copy link
Collaborator

So we can apply the same fix as for nix_build ? that's good news!

think so!

@b-rodrigues
Copy link
Contributor

So on Darwin it works with a non temporary folder?

@philipp-baumann
Copy link
Collaborator

philipp-baumann commented Jul 10, 2024

Seems have the same tempdir problem on any UNIX, because writeLines needs a file that exists for a connection.

> library(rix)
> path_default_nix <- tempdir()
> rix(r_ver = "4.2.1",
+     r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
+     system_pkgs = NULL,
+     git_pkgs = NULL,
+     ide = "other",
+     project_path = path_default_nix,
+     overwrite = TRUE)
Error in file(con, "w") : cannot open the connection
In addition: Warning message:
In file(con, "w") :
  cannot open file '/tmp/RtmpQAja8C/default.nix': No such file or directory
> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Rocky Linux 8.7 (Green Obsidian)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblasp-r0.3.15.so

locale:
 [1] LC_CTYPE=C                 LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rix_0.8.0

loaded via a namespace (and not attached):
[1] compiler_4.2.1   tools_4.2.1      curl_5.2.1       codetools_0.2-18
[5] jsonlite_1.8.8   sys_3.4.2

> dir.exists(path_default_nix)
[1] FALSE

@philipp-baumann
Copy link
Collaborator

philipp-baumann commented Jul 10, 2024

what's surprising why our ubuntu test CI's did not pick this up.

@arraytools could you test this PR with the fix and confirm it works when in any other dir than a tempdir()?: #244 thanks a lot!

@arraytools
Copy link
Author

thanks. I run a test. But I got the same error.

R version 4.4.0 (2024-04-24) -- "Puppy Cup"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(rix)
> path_default_nix <- "~/nix/rix/d1-installing-r-packages"
> rix(r_ver = "4.2.1",
    r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
    system_pkgs = NULL,
    git_pkgs = NULL,
    ide = "other",
    project_path = path_default_nix,
    overwrite = TRUE)
Error in sys::exec_internal(cmd = cmd, args = args) : 
  Executing 'nix-hash' failed with status 1
> traceback()
17: stop(sprintf("Executing '%s' failed with status %d", cmd, status))
16: sys::exec_internal(cmd = cmd, args = args)
15: nix_sri_hash(path = path_to_source_root)
14: hash_url(url = repo_url)
13: hash_cran(repo_url = repo_url)
12: nix_hash(repo_url, commit)
11: get_sri_hash_deps(repo_url, branch_name = NULL, commit = NULL)
10: FUN(X[[i]], ...)
9: lapply(archive_pkgs, fetchzip)
8: paste(lapply(archive_pkgs, fetchzip), collapse = "\n")
7: fetchzips(archive_pkgs)
6: paste(fetchgits(git_pkgs), fetchzips(archive_pkgs), collapse = "\n")
5: fetchpkgs(git_pkgs, archive_pkgs)
4: sprintf("\n  git_archive_pkgs = [%s  ];\n", fetchpkgs(git_pkgs, 
       archive_pkgs))
3: generate_git_archived_pkgs(git_pkgs, cran_pkgs$archive_pkgs, 
       flag_git_archive)
2: paste(generate_header(nix_repo, r_ver, rix_call), generate_rpkgs(cran_pkgs$rPackages, 
       flag_rpkgs), generate_git_archived_pkgs(git_pkgs, cran_pkgs$archive_pkgs, 
       flag_git_archive), generate_tex_pkgs(tex_pkgs), generate_local_r_pkgs(local_r_pkgs, 
       flag_local_r_pkgs), generate_system_pkgs(system_pkgs, r_pkgs), 
       generate_wrapped_pkgs(ide, attrib, flag_git_archive, flag_rpkgs, 
           flag_local_r_pkgs), generate_shell(flag_git_archive, 
           flag_rpkgs, flag_tex_pkgs, flag_local_r_pkgs, flag_wrapper, 
           shell_hook), collapse = "\n")
1: rix(r_ver = "4.2.1", r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"), 
       system_pkgs = NULL, git_pkgs = NULL, ide = "other", project_path = path_default_nix, 
       overwrite = TRUE)
> nix_sri_hash
Error: object 'nix_sri_hash' not found
> rix:::nix_sri_hash
function (path) 
{
    if (!dir.exists(path)) {
        stop("Directory", path, "does not exist", call. = FALSE)
    }
    has_nix_shell <- nix_shell_available()
    if (isFALSE(has_nix_shell)) {
        stop_no_nix_shell()
    }
    LD_LIBRARY_PATH_default <- Sys.getenv("LD_LIBRARY_PATH")
    needs_ld_fix <- nzchar(Sys.getenv("NIX_STORE")) && nzchar(LD_LIBRARY_PATH_default)
    if (isTRUE(needs_ld_fix)) {
        fix_ld_library_path()
    }
    cmd <- "nix-hash"
    args <- c("--type", "sha256", "--sri", path)
    proc <- sys::exec_internal(cmd = cmd, args = args)
    poll_sys_proc_blocking(cmd = paste(cmd, paste(args, collapse = " ")), 
        proc = proc, what = cmd, message_type = "quiet")
    if (isTRUE(needs_ld_fix)) {
        on.exit(Sys.setenv(LD_LIBRARY_PATH = LD_LIBRARY_PATH_default))
    }
    sri_hash <- sys::as_text(proc$stdout)
    return(sri_hash)
}
<bytecode: 0x5564d25ec190>
<environment: namespace:rix>
> file.exists("~/nix/rix/d1-installing-r-packages")
[1] TRUE

@philipp-baumann
Copy link
Collaborator

philipp-baumann commented Jul 11, 2024

thanks. I run a test. But I got the same error.

R version 4.4.0 (2024-04-24) -- "Puppy Cup"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(rix)
> path_default_nix <- "~/nix/rix/d1-installing-r-packages"
> rix(r_ver = "4.2.1",
    r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
    system_pkgs = NULL,
    git_pkgs = NULL,
    ide = "other",
    project_path = path_default_nix,
    overwrite = TRUE)
Error in sys::exec_internal(cmd = cmd, args = args) : 
  Executing 'nix-hash' failed with status 1
> traceback()
17: stop(sprintf("Executing '%s' failed with status %d", cmd, status))
16: sys::exec_internal(cmd = cmd, args = args)
15: nix_sri_hash(path = path_to_source_root)
14: hash_url(url = repo_url)
13: hash_cran(repo_url = repo_url)
12: nix_hash(repo_url, commit)
11: get_sri_hash_deps(repo_url, branch_name = NULL, commit = NULL)
10: FUN(X[[i]], ...)
9: lapply(archive_pkgs, fetchzip)
8: paste(lapply(archive_pkgs, fetchzip), collapse = "\n")
7: fetchzips(archive_pkgs)
6: paste(fetchgits(git_pkgs), fetchzips(archive_pkgs), collapse = "\n")
5: fetchpkgs(git_pkgs, archive_pkgs)
4: sprintf("\n  git_archive_pkgs = [%s  ];\n", fetchpkgs(git_pkgs, 
       archive_pkgs))
3: generate_git_archived_pkgs(git_pkgs, cran_pkgs$archive_pkgs, 
       flag_git_archive)
2: paste(generate_header(nix_repo, r_ver, rix_call), generate_rpkgs(cran_pkgs$rPackages, 
       flag_rpkgs), generate_git_archived_pkgs(git_pkgs, cran_pkgs$archive_pkgs, 
       flag_git_archive), generate_tex_pkgs(tex_pkgs), generate_local_r_pkgs(local_r_pkgs, 
       flag_local_r_pkgs), generate_system_pkgs(system_pkgs, r_pkgs), 
       generate_wrapped_pkgs(ide, attrib, flag_git_archive, flag_rpkgs, 
           flag_local_r_pkgs), generate_shell(flag_git_archive, 
           flag_rpkgs, flag_tex_pkgs, flag_local_r_pkgs, flag_wrapper, 
           shell_hook), collapse = "\n")
1: rix(r_ver = "4.2.1", r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"), 
       system_pkgs = NULL, git_pkgs = NULL, ide = "other", project_path = path_default_nix, 
       overwrite = TRUE)
> nix_sri_hash
Error: object 'nix_sri_hash' not found
> rix:::nix_sri_hash
function (path) 
{
    if (!dir.exists(path)) {
        stop("Directory", path, "does not exist", call. = FALSE)
    }
    has_nix_shell <- nix_shell_available()
    if (isFALSE(has_nix_shell)) {
        stop_no_nix_shell()
    }
    LD_LIBRARY_PATH_default <- Sys.getenv("LD_LIBRARY_PATH")
    needs_ld_fix <- nzchar(Sys.getenv("NIX_STORE")) && nzchar(LD_LIBRARY_PATH_default)
    if (isTRUE(needs_ld_fix)) {
        fix_ld_library_path()
    }
    cmd <- "nix-hash"
    args <- c("--type", "sha256", "--sri", path)
    proc <- sys::exec_internal(cmd = cmd, args = args)
    poll_sys_proc_blocking(cmd = paste(cmd, paste(args, collapse = " ")), 
        proc = proc, what = cmd, message_type = "quiet")
    if (isTRUE(needs_ld_fix)) {
        on.exit(Sys.setenv(LD_LIBRARY_PATH = LD_LIBRARY_PATH_default))
    }
    sri_hash <- sys::as_text(proc$stdout)
    return(sri_hash)
}
<bytecode: 0x5564d25ec190>
<environment: namespace:rix>
> file.exists("~/nix/rix/d1-installing-r-packages")
[1] TRUE

sorry there was a wrong check, the fixing needs to be done in a non-Nix R session. With this, it should work:

this PR does make it work for me:
#244

@philipp-baumann
Copy link
Collaborator

@arraytools i should have fixed both the tempdir issue and also the nix-shell issue. Can you also test again with tempdir? thanks!

> path_default_nix <- tempdir()
> rix(r_ver = "4.2.1",
+     r_pkgs = c("dplyr@0.8.0", "janitor@1.0.0"),
+     system_pkgs = NULL,
+     git_pkgs = NULL,
+     ide = "other",
+     project_path = path_default_nix,
+     overwrite = TRUE)

### Bootstrapping isolated, project-specific, and runtime-pure R setup via Nix ###

==> Existing isolated nix-R project folder:
 /tmp/RtmpUBtSix 

* current R session running outside Nix environment and not from RStudio

==> Added `.Rprofile` file and code lines for new R sessions launched from:
/tmp/RtmpUBtSix

* Added the location of the Nix store to `PATH` environmental variable for new R sessions on host/docker RStudio:
/nix/var/nix/profiles/default/bin

==> Also adjusting `PATH` via `Sys.setenv()`, so that system commands can invoke key Nix commands like `nix-build` in this RStudio session outside Nix

### Successfully generated `default.nix` and `.Rprofile` ###


> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Rocky Linux 8.7 (Green Obsidian)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblasp-r0.3.15.so

locale:
 [1] LC_CTYPE=C                 LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rix_0.8.0       testthat_3.1.10

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.11       compiler_4.2.1    later_1.3.1       urlchecker_1.0.1 
 [5] sys_3.4.2         prettyunits_1.1.1 profvis_0.3.8     remotes_2.4.2.1  
 [9] tools_4.2.1       digest_0.6.33     pkgbuild_1.4.2    pkgload_1.3.2.1  
[13] jsonlite_1.8.8    memoise_2.0.1     lifecycle_1.0.3   rlang_1.1.1      
[17] shiny_1.7.5       cli_3.6.2         rstudioapi_0.15.0 curl_5.2.1       
[21] fastmap_1.1.1     withr_2.5.0       stringr_1.5.0     desc_1.4.2       
[25] fs_1.6.3          htmlwidgets_1.6.2 vctrs_0.6.3       devtools_2.4.5   
[29] rprojroot_2.0.3   glue_1.6.2        R6_2.5.1          processx_3.8.2   
[33] sessioninfo_1.2.2 callr_3.7.3       purrr_1.0.2       magrittr_2.0.3   
[37] codetools_0.2-18  ps_1.7.5          promises_1.2.1    ellipsis_0.3.2   
[41] htmltools_0.5.6   usethis_2.2.2     mime_0.12         xtable_1.8-4     
[45] httpuv_1.6.11     stringi_1.7.12    miniUI_0.1.1.1    cachem_1.0.8     
[49] crayon_1.5.2      brio_1.1.3

@philipp-baumann philipp-baumann added the bug Something isn't working label Jul 11, 2024
@b-rodrigues
Copy link
Contributor

b-rodrigues commented Jul 11, 2024

I understand why it worked for me: I wasn't using a tempdir() 🤦‍♂️

With a tempdir() it also failed for me on openSUSE. Can confirm your fix works!

@arraytools
Copy link
Author

The last fix passes both the tempdir() and existing dir cases. Great job!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants