Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade spectrum-mpi module on Summit #3114

Closed
dqwu opened this issue Aug 6, 2019 · 2 comments · Fixed by #3115
Closed

Upgrade spectrum-mpi module on Summit #3114

dqwu opened this issue Aug 6, 2019 · 2 comments · Fixed by #3115

Comments

@dqwu
Copy link
Contributor

dqwu commented Aug 6, 2019

On Summit, the current default spectrum-mpi module is:
spectrum-mpi/10.3.0.1-20190611

PS, "module avail spectrum-mpi" shows this version only.

If we purge this latest version and load spectrum-mpi/10.3.0.0-20190419,
"module list" marks it as a hidden module:

spectrum-mpi/10.3.0.0-20190419 (H)
  Where:
   H:  Hidden Module

This suggests that spectrum-mpi/10.3.0.0-20190419 is deprecated on Summit.
However, it is still the specified version in latest config_machines.xml:

      <modules compiler="ibm" mpilib="!mpi-serial">
	    <command name="load">spectrum-mpi/10.3.0.0-20190419</command>
      </modules>
      <modules compiler="pgi.*" mpilib="!mpi-serial">
	    <command name="load">spectrum-mpi/10.3.0.0-20190419</command>
      </modules>
      <modules compiler="gnu" mpilib="!mpi-serial">
	    <command name="load">spectrum-mpi/10.3.0.0-20190419</command>
      </modules>

If spectrum-mpi/10.3.0.0-20190419 is used, a low-res F case (e.g. --compset FC5AV1C-H01B --res ne4_ne4) fails on Summit:
SIGSEGV - Segmentation violation in procedure ncmpiio_open

Switching to spectrum-mpi/10.3.0.1-20190611 seems to work.

@whannah1
Copy link
Contributor

whannah1 commented Aug 6, 2019

I've had luck by removing the version completely when running with the ECP fork. I also had to remove the versions from other libraries and that seemed to fix some issues we were having.

sarats added a commit that referenced this issue Aug 6, 2019
Default MPI module has been updated on Summit. Old module is still
available but hidden in listing.

A SIGSEGV error was encountered using older module with F-case. This
module update fixes that issue.

Fixes #3114

[BFB]
@sarats
Copy link
Member

sarats commented Aug 6, 2019

We explicitly mention software versions in master to record what was known to work (last known good configuration) and keep track of changes in software environment.

minxu74 added a commit that referenced this issue Aug 9, 2019
Default MPI module has been updated on Summit. Old module is still
available but hidden in listing.

A SIGSEGV error was encountered using older module with F-case. This
module update fixes that issue.

Fixes #3114

[BFB]

* sarats/machinefiles/summit-mpi-aug-2019:
  Summit machine file: Update MPI module
minxu74 added a commit that referenced this issue Aug 9, 2019
Default MPI module has been updated on Summit. Old module is still
available but hidden in listing.

A SIGSEGV error was encountered using older module with F-case. This
module update fixes that issue.

Fixes #3114

[BFB]
tanzeli1982 pushed a commit that referenced this issue Aug 12, 2019
Default MPI module has been updated on Summit. Old module is still
available but hidden in listing.

A SIGSEGV error was encountered using older module with F-case. This
module update fixes that issue.

Fixes #3114

[BFB]
jgfouca added a commit that referenced this issue Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants