Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generating container fails #71

Closed
pelesh opened this issue Nov 19, 2023 · 3 comments · Fixed by #72
Closed

Generating container fails #71

pelesh opened this issue Nov 19, 2023 · 3 comments · Fixed by #72
Assignees
@cameronrutherford @jaelynlitz
Labels
bug Something isn't working
Milestone
1.6.1 Python bind...

Comments

@pelesh
Copy link
Collaborator

pelesh commented Nov 19, 2023 •
edited
Loading

Issue type

  • Bug

Relates to

  • CMake build system
  • Spack configuration
  • Web docs
  • Other

Summary

Generating docker image following ExaGO instructions fails.

Exact commands to reproduce, if applicable

On MacOS arch follow instructions to the letter.

Relevant logs and/or screenshots, if applicable

Error output: 
=> => transferring context: 56B                                           0.0s
 => [builder  2/13] RUN apt-get -yqq update && apt-get -yqq upgrade  &&   14.2s
 => [builder  3/13] RUN mkdir /opt/spack-environment &&  (echo spack: &&   0.5s
 => [builder  4/13] COPY coinhsl-archive-2019.05.21.tar.gz /opt/spack-env  0.0s
 => [builder  5/13] RUN cd /opt/spack-environment && spack env activate .  6.0s
 => [builder  6/13] RUN cd /opt/spack-environment && spack env activate  521.3s
 => [builder  7/13] RUN cd /opt/spack-environment && spack env activat  1112.6s 
 => ERROR [builder  8/13] RUN cd /opt/spack-environment && spack env ac  140.1s 
------                                                                          
 > [builder  8/13] RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc:
3.979 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libiconv-1.17-iga2iuwmbar5rlubtrf3ptsa2a7o3uzm
3.979 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/gmake-4.4.1-75r7yzw3krhytcs7xqptozjuxw6zdbiv
3.979 ==> Installing xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq [3/47]
3.979 ==> No binary for xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq found: installing from source
4.483 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/dd/dd172acb53867a68012f94c17389401b2f274a1aa5ae8f84cbfb8b7e383ea8d3.tar.bz2
4.483 ==> No patches needed for xz
4.489 ==> xz: Executing phase: 'autoreconf'
4.495 ==> xz: Executing phase: 'configure'
12.70 ==> xz: Executing phase: 'build'
21.38 ==> xz: Executing phase: 'install'
23.19 ==> xz: Successfully installed xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq
23.19   Stage: 0.36s.  Autoreconf: 0.00s.  Configure: 8.20s.  Build: 8.68s.  Install: 1.69s.  Post-install: 0.09s.  Total: 19.05s
23.20 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq
24.14 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/nghttp2-1.57.0-6jehmemf3s5usdiqtkej3ydoyahy3s6r
24.14 ==> Installing libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l [5/47]
24.14 ==> No binary for libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l found: installing from source
24.35 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/d6/d66c56ad259a82cf2a9dfc408b32bf5da52371500b84745f7fb8b645712df676.tar.gz
24.35 ==> No patches needed for libffi
24.35 ==> libffi: Executing phase: 'autoreconf'
24.36 ==> libffi: Executing phase: 'configure'
30.39 ==> libffi: Executing phase: 'build'
31.74 ==> libffi: Executing phase: 'install'
32.08 ==> libffi: Successfully installed libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l
32.08   Stage: 0.06s.  Autoreconf: 0.00s.  Configure: 6.02s.  Build: 1.34s.  Install: 0.27s.  Post-install: 0.05s.  Total: 7.78s
32.08 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l
33.16 ==> Installing findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r [6/47]
33.16 ==> No binary for findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r found: installing from source
33.52 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/a2/a2bfb8c09d436770edc59f50fa483e785b161a3b7b9d547573cb08065fd462fe.tar.xz
33.52 ==> Applied patch /opt/spack/var/spack/repos/builtin/packages/findutils/nonnull.patch
33.52 ==> findutils: Executing phase: 'autoreconf'
33.53 ==> findutils: Executing phase: 'configure'
71.89 ==> findutils: Executing phase: 'build'
76.50 ==> findutils: Executing phase: 'install'
77.90 ==> findutils: Successfully installed findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r
77.90   Stage: 0.22s.  Autoreconf: 0.00s.  Configure: 38.35s.  Build: 4.61s.  Install: 1.26s.  Post-install: 0.07s.  Total: 44.55s
77.90 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r
78.95 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/zlib-ng-2.1.4-yigb7oxo5yoy6lfcreefhio5bprbrfod
78.95 ==> Installing libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3 [8/47]
78.95 ==> No binary for libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3 found: installing from source
79.12 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/cd/cdac3941803364cf81a908499beb79c200ead60b6b5b40cad124fd1e06caa295.tar.gz
79.12 ==> No patches needed for libsigsegv
79.13 ==> libsigsegv: Executing phase: 'autoreconf'
79.13 ==> libsigsegv: Executing phase: 'configure'
83.84 ==> libsigsegv: Executing phase: 'build'
84.71 ==> libsigsegv: Executing phase: 'install'
84.91 ==> libsigsegv: Successfully installed libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3
84.91   Stage: 0.03s.  Autoreconf: 0.00s.  Configure: 4.70s.  Build: 0.86s.  Install: 0.13s.  Post-install: 0.05s.  Total: 5.82s
84.91 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3
85.89 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/berkeley-db-18.1.40-bw4gbyq362zopdk7r2yiiqp2wmkxjdfh
85.89 ==> Installing libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c [10/47]
85.89 ==> No binary for libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c found: installing from source
86.09 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/f5/f51c921042e34beddeded4b75557656559cf5b1f2448033b4c1eec11c07e530f.tar.xz
86.09 ==> No patches needed for libmd
86.09 ==> libmd: Executing phase: 'autoreconf'
86.14 ==> libmd: Executing phase: 'configure'
90.70 ==> libmd: Executing phase: 'build'
91.77 ==> libmd: Executing phase: 'install'
92.08 ==> libmd: Successfully installed libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c
92.08   Stage: 0.04s.  Autoreconf: 0.00s.  Configure: 4.59s.  Build: 1.06s.  Install: 0.24s.  Post-install: 0.05s.  Total: 6.02s
92.08 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c
93.03 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/metis-5.1.0-7azwzl2xgjxrin2lqaxptj2d3jrzgxuv
93.03 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/bzip2-1.0.8-in57fqfaz6hyaqelvctbac2ytafuxwlk
93.03 ==> Installing zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j [13/47]
93.03 ==> No binary for zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j found: installing from source
93.29 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/98/98e9c3d949d1b924e28e01eccb7deed865eefebf25c2f21c702e5cd5b63b85e1.tar.gz
93.29 ==> No patches needed for zstd
93.30 ==> zstd: Executing phase: 'edit'
93.31 ==> zstd: Executing phase: 'build'
93.32 ==> zstd: Executing phase: 'install'
136.4 ==> zstd: Successfully installed zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j
136.4   Stage: 0.09s.  Edit: 0.00s.  Build: 0.00s.  Install: 42.97s.  Post-install: 0.05s.  Total: 43.16s
136.4 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j
137.5 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/openblas-0.3.25-yhvdc5frvmfruqnaivudtzq5marlygvp
137.5 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/diffutils-3.9-nggcs7jmd7jtwokiva2xkfbxqktb2a4b
137.5 ==> Installing util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2 [16/47]
137.5 ==> No binary for util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2 found: installing from source
137.7 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/0f/0f812e6e9d2786ba8f54b960ee563c0663ddbe2434bf24ff193f5feab1f31971.tar.bz2
137.7 ==> No patches needed for util-macros
137.7 ==> util-macros: Executing phase: 'autoreconf'
137.7 ==> util-macros: Executing phase: 'configure'
139.1 ==> util-macros: Executing phase: 'build'
139.1 ==> util-macros: Executing phase: 'install'
139.2 ==> util-macros: Successfully installed util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2
139.2   Stage: 0.02s.  Autoreconf: 0.00s.  Configure: 1.34s.  Build: 0.00s.  Install: 0.02s.  Post-install: 0.04s.  Total: 1.48s
139.2 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2
139.6 ==> Error: Couldn't find patch for package builtin.libffi with sha256: 070b1f3aa87f2b56f83aff38afc42157e1692bfaa580276ecdbad2048b818ed7. This usually means the patch was modified or removed. To fix this, either reconcretize or use the original package repository
139.6 ==> Updating view at /opt/views/view
------
Dockerfile-with-features:53
--------------------
  51 |     
  52 |     # Install PETSc 
  53 | >>> RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc
  54 |     
  55 |     # Install py-jupyterlab
--------------------
ERROR: failed to solve: process "docker-shell cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc" did not complete successfully: exit code: 1
[1797783 ms] Error: Command failed: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /var/folders/b9/64xrxgrn12z891ttr3856zb1d4rlq3/T/devcontainercli/container-features/0.52.1-1700330517994/Dockerfile-with-features -t vsc-exago-c8cb40076f187c020704a3ecb671fc06d1aca656f975162185c3d1431ac55d34 --target dev_containers_target_stage --no-cache --pull --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /Users/55y/src/exago/exago/ExaGO
[1797784 ms]     at cAA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:463:1860)
[1797784 ms]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[1797784 ms]     at async zw (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:462:1841)
[1797784 ms]     at async kK (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:462:610)
[1797784 ms]     at async NAA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:479:3660)
[1797784 ms]     at async vC (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:479:4775)
[1797784 ms]     at async etA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:613:10941)
[1797784 ms]     at async AtA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:613:10682)
[1797806 ms] Exit code 1
[1797815 ms] Command failed: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/55y/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-0b0ea4b4-bacc-4dc3-bd79-4ca08b0198481700330515815 --workspace-folder /Users/55y/src/exago/exago/ExaGO --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/55y/src/exago/exago/ExaGO --id-label devcontainer.config_file=/Users/55y/src/exago/exago/ExaGO/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/55y/src/exago/exago/ExaGO/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --build-no-cache --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[1797815 ms] Exit code 1

Possible fixes, if applicable

No obvious fixes.
@pelesh pelesh added the bug Something isn't working label Nov 19, 2023
@cameronrutherford
Copy link
Contributor

Unforunately we will need more of the error output to really know what's going on here, and how to fix it.

Key is that we need to see what the concrete spec actually was, and then we can figure out a way to avoid this buggy dependency. We have also merged quite a few things into develop. I am rebuilding myself now, and am also going to ensure the viz frontend is built already into the container.

This will also greatly simplify example notebook, so I will clean that up as well.

@cameronrutherford
Copy link
Contributor

@pelesh I was able to reproduce on my machine, and this issue seems to have originated from spack/spack#41083

A temporary fix is I am now building with libffi@3.4.2, but I think this warrants a spack issue / comment on that PR.

I didn't look too hard to see if Spack PR already exists, but this change is 5 days old, so I think we happened to catch an edge case here...

@cameronrutherford
Copy link
Contributor

This does motivate pinning of a spack version within the container that we are building, and would perhaps need some help from @rothpc and techniques discussed offline in order to maintain exago's package alongside the pinned version of Spack that won't be getting regular updates.

Then again it is nice to keep up to date with spack develop, and I think #52 and CI/CD to cache a build of this devconatiner will help a lot.

Unfortunately this build would not be with Ipopt until we can build ipopt+resolve.

@cameronrutherford cameronrutherford mentioned this issue Nov 20, 2023
Merged
5 tasks
@cameronrutherford cameronrutherford mentioned this issue Nov 21, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cameronrutherford cameronrutherford

@jaelynlitz jaelynlitz

Labels
bug Something isn't working
Projects
None yet
Milestone
1.6.1 Python bindings + Documentation...
Development

Successfully merging a pull request may close this issue.

Add nvm install to Dockerfile and bugfix libffi. pnnl/ExaGO
3 participants
@pelesh @cameronrutherford @jaelynlitz