Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NERSC performance archiving for multiple projects #5753

Open
sarats opened this issue Jun 9, 2023 · 5 comments
Open

NERSC performance archiving for multiple projects #5753

sarats opened this issue Jun 9, 2023 · 5 comments
Labels
Machine Files Performance pm-cpu Perlmutter at NERSC (CPU-only nodes) pm-gpu Perlmutter machine at NERSC (GPU nodes)

Comments

@sarats
Copy link
Member

sarats commented Jun 9, 2023

<SAVE_TIMING_DIR_PROJECTS>e3sm,m3411,m3412</SAVE_TIMING_DIR_PROJECTS>

We can use a wildcard like '*' but that probably would capture a lot of experiments from non-E3SM project members.
We need to think if we would still like to capture all E3SM runs on Perlmutter.

Otherwise, we need to update the list of projects with any others that we wish to archive.

The incremental cost of allowing additional projects to be archived is proportional to the number of jobs they plan to run. At this stage, I want to evaluate this on a case by case basis.

@sarats sarats added Machine Files Performance pm-gpu Perlmutter machine at NERSC (GPU nodes) pm-cpu Perlmutter at NERSC (CPU-only nodes) labels Jun 9, 2023
@sarats
Copy link
Member Author

sarats commented Jun 9, 2023

Will add m4259 that is used by ImPACTS SciDAC project
cc @vanroekel

@darincomeau
Copy link
Member

m1199 for the E3SM-Arctic simulations under the HiLAT-RASM project would be great to have.
cc: @milenaveneziani

@sarats
Copy link
Member Author

sarats commented Jun 10, 2023

Other projects that were requested.

m3520 cc: @bishtgautam RDycore SciDAC

m4048 from @beharrop

Wei Cheng is the PI. It's an RGMA project looking at ECS sensitivity to ocean heat transport and cloud feedbacks using the slab ocean configuration of E3SM with a cloud-locking capability

@sarats
Copy link
Member Author

sarats commented Jun 10, 2023

m3312 cc: @whannah1 PI & topic?

@ndkeen Do you know the topics and PIs for m3411 and m3412 (existing)?

@sarats
Copy link
Member Author

sarats commented Jun 10, 2023

One aspect to consider is the performance archive location and ability for users in some of these projects to write to that location.

Currently, it's set to
<SAVE_TIMING_DIR>/global/cfs/cdirs/e3sm</SAVE_TIMING_DIR>

We ought to consider a world-writable location in that case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Machine Files Performance pm-cpu Perlmutter at NERSC (CPU-only nodes) pm-gpu Perlmutter machine at NERSC (GPU nodes)
Projects
None yet
Development

No branches or pull requests

2 participants